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INCORPORATING  LESSONS  LEARNED  INTO  THE  ARMY  COMPETENCY 
ASSESSMENT  PROTOTYPE 

EXECUTIVE  SUMMARY 


Research  Requirement: 

The  Army  is  changing  to  meet  the  needs  of  the  21st  century,  which  requires  that  all 
Soldiers  possess  the  knowledge,  skills,  and  other  attributes  (KSAs)  that  will  enable  them  to 
perform  effectively  in  complex  technical  environments,  under  multiple  and  changing  mission 
requirements,  and  in  semi-autonomous,  widely  dispersed  teams.  The  Army  needs  an  integrated 
Soldier  assessment  system  to  support  these  changes,  and  the  PerformM21  project  is  an  attempt  to 
answer  this  call.  The  Lessons  Learned  project  is  part  of  the  larger  PerformM21  body  of  work  that 
was  undertaken  to  show  how  lessons  learned  from  new  deployments  can  be  incorporated  into  the 
existing  Army-wide  Common  Tasks  structure. 

Procedure: 

This  10- mo  nth  project  began  with  the  identification  of  doctrinal  and  training  sources  of 
lessons  learned  in  Operation  Iraqi  Freedom/Operation  Enduring  Freedom  (OIF/OEF).  Initially  these 
sources  provided  the  basis  of  knowledges  and  tasks  used  in  a  job  analysis  survey  targeting  the 
performance  of  E4  Soldiers  ready  for  promotion  to  E5.  Based  on  a  review  of  job  analysis  research, 
scales  were  developed,  and  the  survey  was  administered  to  53  E4  Soldiers  and  22  non-commissioned 
officers  (NCOs).  The  results  of  the  survey  were  used  to  develop  a  prototype  test  blueprint  that 
incorporates  current.  Army-wide  Common  Tasks  with  “lessons  learned”  knowledges  and  tasks.  The 
sources  of  lessons  learned  subsequently  served  as  reference  material  for  new  item  content. 

Findings: 

The  product  of  this  research  is  the  process  used  to  locate  and  integrate  lessons  learned 
from  recent  deployments  with  existing  Army  common  tasks.  A  number  of  sources  were 
identified  for  lessons  learned  content,  but  there  were  challenges  including  locating  (a)  actual 
doctrine  as  opposed  to  informal  guides,  and  (b)  information  applicable  across  time  and  location. 
An  operational  program  would  require  up-to-date  doctrine,  both  for  test  developers  and  Soldiers. 
The  survey  proved  to  be  an  efficient  way  to  analyze  and  combine  feedback  from  a  variety  of 
Soldiers  to  create  a  prototype  test  blueprint.  The  survey  permitted  the  development  of  Army¬ 
wide  assessment  items  addressing  these  lessons  learned  that  would  otherwise  be  absent  from  the 
comprehensive  competency  assessment  developed  for  the  PerformM21  project. 

Utilization  and  Dissemination  of  Findings: 

The  approach  we  took  in  locating  lessons  learned  sources  will  be  valuable  to  Army  leaders 
who  are  considering  an  operational  testing  environment.  The  crux  of  the  dilemma  is  information; 
how  to  get  it,  how  to  disseminate  it,  and  how  to  ensure  it  is  properly  vetted  doctrine.  The  survey 
development  work  is  directly  translatable  into  an  operational  program.  We  suggest  collecting  further 
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field  input,  but  expect  the  changes  to  be  fairly  minor.  In  fact,  collecting  data  from  a  larger  sample 
would  provide  the  Army  with  a  reasonable  working  operational  test  blueprint. 
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CHAPTER  1:  INTRODUCTION  AND  OVERVIEW  OF  RESEARCH 

The  Army  is  changing  to  meet  the  needs  of  the  21st  century.  All  Soldiers  must  possess 
the  knowledge,  skills,  and  other  attributes  (KSAs)  that  will  enable  them  to  perform  effectively  in 
complex  technical  environments,  under  multiple  and  changing  mission  requirements,  and  in 
semi-autonomous,  widely  dispersed  teams.  The  Army  needs  an  integrated  Soldier  assessment 
system  to  support  these  changes. 

The  PerformM21  project  is  attempting  to  answer  the  call  through  the  development  of 
Army-wide  and  military  occupational  specialty  (MOS)-specific  prototype  assessments.  The 
PerformM21  effort  is  driven  by  the  vision  of  an  integrated  selection,  assignment,  and 
performance  assessment  system  that  (a)  meets  current  and  future  force  requirements;  (b)  is  open 
to  work  and  job  changes;  and  (c)  applies  cost  effective  technologies  for  system  maintenance, 
effectiveness,  and  use  by  commanders,  personnel  decision  makers,  trainers,  and  developers. 
Another  influence  is  the  need  identified  by  the  Army  Training  and  Leader  Development  Panel 
(ATLDP)  report  of  April  2002.  In  the  report,  recommendations  were  to  provide  Soldiers  with 
self-assessment  and  institutional  assessment  of  job  performance  capabilities.  Furthermore,  Army 
leadership  and  senior  non-commissioned  officers  (NCOs)  have  placed  emphasis  on  providing  an 
assessment  prototype  in  a  timely  manner  to  demonstrate  commitment  to  the  change  and 
improvement  process. 

The  PerformM21  project  was  conducted  as  a  three-phase  effort.  In  the  first  phase, 
researchers  (a)  examined  historical  issues  associated  with  competency  testing  and  identified  21st 
century  solutions  to  these  issues,  (b)  identified  assessment  alternatives,  and  (c)  worked  with 
senior  NCOs  across  the  Army  to  develop  a  prototype  Army-wide  knowledge  assessment.  In  the 
second  phase,  researchers  evaluated  the  prototype  job  knowledge  assessment  with  data 
collections  at  a  variety  of  Army  installations,  and  developed  prototype  competency  assessments 
for  five  MOS:  14E  -  Patriot  Missile  Operator/Maintainer,  19K  -  Ml  Armor  Crewman,  31B  - 
Military  Police,  63B  -  Wheeled  Vehicle  Mechanic,  and  91W  -  Health  Care  Specialist.  Phase  III 
of  the  project  provided  recommendations  on  the  feasibility  of  introducing  a  new  Soldier 
assessment  system  based  on  the  prototype  Army-wide  and  MOS-specific  test  development  and 
evaluation  work.  Included  in  this  phase  was  cost  analysis  work  that  was  designed  to  extrapolate 
from  this  effort  to  a  larger  system  of  Army-wide  and  MOS  testing. 

Since  the  initiation  of  the  PerformM21  project,  lessons  learned  from  Operation  Iraqi 
Freedom/Operation  Enduring  Freedom  (OIF/OEF)  have  emerged.  The  PerformM21  Army  Test 
Program  Advisory  Team  (ATP AT),  a  group  of  senior  NCOs  who  provide  guidance  and  support 
to  the  PerformM21  researchers,  encouraged  the  development  of  new  assessment  items  to  reflect 
these  combat  critical  tasks  and  knowledges.  Along  with  development  of  new  items,  of  particular 
interest  to  us  was  the  process  by  which  new  lessons  learned  tasks  and  knowledges  would  be 
incorporated  into  existing  Army  tasks  and  knowledges.  This  is  not  as  straight-forward  a  process 
as  it  seems.  Consider  the  following  factors: 
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•  From  where,  exactly,  do  these  “lessons”  come? 

•  Who  is  the  Final  arbiter  of  a  task  or  knowledge  being  declared  a  lesson  learned? 

•  Many  lessons  are  likely  to  be  theater-specific. 

•  Some  lessons  might  conflict  with  established  Army  doctrine. 

•  There  is  likely  to  be  little  established  doctrine  for  these  lessons  to  support  item  or 
training  development. 

This  research  was  intended  to  complement  the  research  from  the  PerformM21  project  by 
addressing  these  and  other  issues.  In  carrying  out  this  research,  to  be  consistent  with 
PerformM21,  we  focused  on  E4  Soldiers  ready  for  promotion  to  E5. 


_ Step  6:  Develop  test  items 

Figure  1.  Research  process  overview 


We  accomplished  the  study  by  following  the 
steps  in  Figure  1 .  We  identified  doctrinal  and  training 
sources  of  lessons  learned  in  OIF/OEF,  developed 
and  conducted  a  knowledge  and  task  survey,  created  a 
prototype  test  blueprint,  and  developed  prototype  test 
items.  The  remaining  chapters  describe  these  steps  in 
more  detail.  Chapter  2  describes  the  process  followed 
for  completing  the  first  step.  It  also  discusses  the 
issues  we  confronted,  such  as  lack  of  doctrine,  for 
official  use  only  (FOUO)  classification  of  many 
sources,  and  the  theater-specificity  of  some  of  the 
lessons.  The  second  through  fifth  steps  are  reviewed 
in  Chapter  3.  We  discuss  the  issues  the  Army  needs  to 
consider  when  creating  an  operational  job  analysis 
survey  both  in  general  and  to  address  emerging 
lessons  learned.  The  sixth  step  is  covered  in  Chapter 
4.  We  used  the  sources  of  the  lessons  learned  to 
develop  new  item  content.  Finally,  Chapter  5  contains 
the  summary  of  our  findings  in  completing  the  work. 
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CHAPTER  2:  IDENTIFICATION  OF  SOLDIER  REQUIREMENTS 

This  chapter  documents  the  developmental  efforts  to  identify  content  for  Soldier 
requirements  evaluation  focused  on  lessons  learned  from  recent  deployments.  Four  primary 
sources  were  incorporated,  and  a  composite  taxonomy  was  prepared.  The  goal  was  to  develop  a 
categorization  that  would  support  both  near-term  and  long-term  goals,  including  a  Soldier  job 
analysis  survey  and  development  of  a  test  blueprint  that  would  serve  to  support  test  development 
over  several  iterations.  The  discussion  also  considers  the  inherent  problems  in  developing  source 
documentation  that  is  both  current  and  lasting. 

Sources  for  Lessons  Learned 

The  purpose  driving  the  Lessons  Learned  project  was  to  develop  a  test  that  would  reflect 
the  realities  of  Soldier  deployments,  primarily  those  in  support  of  Operation  Iraqi  Freedom  (OIF) 
in  Iraq  and  Operation  Enduring  Freedom  (OEF)  in  Afghanistan.  Performance  and  knowledge 
requirements  were  to  be  Army-wide;  that  is,  not  MOS,  assignment,  or  unit  specific.  Additionally, 
the  work  was  intended  to  supplement  the  earlier  analysis  and  test  development  work  that  had 
been  performed  as  part  of  the  development  of  the  Army-wide  assessment  in  the  earlier  phases  of 
the  PerformM21  project  (R.C.  Campbell,  Keenan,  Moriarty,  Knapp,  &  Heffner,  2004).  That 
work  produced  both  a  task  list  and  test  blueprint  that  constitute  a  core  list  of  performance  and 
knowledge  requirements  as  well  as  a  battery  of  prototype  test  items.  The  content  of  the  Army¬ 
wide  task  list  was  produced  partially  in  response  to  guidance  from  then  Sergeant  Major  of  the 
Army  (SMA)  Tilley  and  reflects  his  direction  that  an  Army  test  should  include,  in  addition  to 
common  skills  testing,  evaluations  of  leadership  and  training  skills,  and  knowledge  of  Army 
history,  customs,  and  mores.  As  such,  it  differs  from  many  other  extant  task  lists,  which  are 
generally  limited  to  common  procedural  areas  and  tasks.  Because  this  task  domain  is  essential  to 
the  Lessons  Learned  project,  a  synopsis  of  its  content  is  provided  in  Appendix  A. 

Also  in  keeping  with  the  tenets  of  the  earlier  work,  the  focus  for  the  Lessons  Learned 
project  was  targeted  towards  Soldiers  at  the  senior  E4  level  being  considered  for  advancement  to 
paygrade  E5.  One  goal  of  the  project  was  to  develop  more  prototype  test  items,  but  the  procedure 
would  follow  the  required  steps  of  identifying  and  defining  the  performance  domain,  to  be 
followed  by  blueprint  development  (based  on  survey  input),  and  finally  the  actual  development  of 
items.  In  defining  the  Lessons  Learned  performance  domain,  there  were  four  additional  sources 
identified: 

•  Warrior  Tasks  and  Battle  Drills 

•  TRADOC-ARI  Common  Tasks  Survey 

•  The  Center  for  Army  Lessons  Learned  (CALL) 

•  Policy,  Command,  and  Authority 

Warrior  Tasks  and  Battle  Drills 

In  September  2003,  the  new  Army  Chief  of  Staff  (ACOS)  initiated  a  review  of  Soldier 
preparation  for  the  War  on  Terrorism.  One  of  his  concerns  was  that  Soldiers  have  training  on 
requisite  tasks  during  initial  entry  training  (IET)  that  would  prepare  them  to  be  deployed 
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immediately  following  such  training  into  combat  situations.  The  initiative,  called  Task  Force 
Soldier,  involved  the  appointment  of  a  Warrior  Task  Site  Selection  Board  that  included  Active 
and  Reserve  Component  officer  and  NCO  subject  matter  experts  (SMEs),  government  employees 
and  contractors,  and  retired  senior  officers  and  NCOs.  This  Board  considered  a  list  of  about  400 
tasks  derived  from  97  MOS  task  lists  and  included  those  tasks  required  of  Soldiers  in  Skill  Level 
(SL)  1  (E1-E4),  SL  2  (E5),  and  SL  3  (E6).  Initially,  the  Board  recommended  the  integration  of 
174  tasks  into  the  Army-wide  lists.  This  was  eventually  reduced  to  39  tasks  and  9  battle  drills. 
The  tasks  are  all  individual  Soldier  performance  requirements.  Battle  drills  are  collective 
requirements,  involving  the  integrated  performance  of  several  Soldiers  and  usually  with  the 
incorporation  of  several  individual  tasks. 

The  process  was  completed  in  early  2004  and  pilot  implantation  in  IET  started 
immediately.  Although  commonly  referred  to  as  “the  39  and  9,”  the  Army  Training  and  Doctrine 
Command  (TRADOC)  rejects  the  assignment  of  a  finite  number  to  the  list  and  maintains  that  the 
Warrior  Task  list  is  a  dynamic  document,  based  on  lessons  learned  that  are  being  acquired  out  of 
the  operational  theaters,  with  both  the  content  of  the  list  and  numbers  of  tasks  and  battle  drills 
subject  to  change  and  revision.  The  current  list  is  included  in  Appendix  B. 

Compounding  the  issue  is  that  not  all  of  the  lessons  learned-generated  changes  in  IET  are 
reflected  in  the  Warrior  Task  and  Battle  Drills  list.  For  example,  in  the  spring  of  2005,  TRADOC 
completed  a  trial  block  of  IET  instruction  based  on  the  introduction  of  a  new  individual  first  aid 
kit.  As  a  result,  several  “new”  IET  tasks  were  added: 

•  Employ  the  Emergency  trauma  dressing  (replaced  the  field  dressing  and  pressure 
dressing  training). 

•  Apply  the  combat  application  tourniquet  (CAT). 

•  Apply  the  large-bore  needle  to  a  compressed  lung  casualty. 

•  Apply  chest  compression  technique  to  a  sucking  chest  wound. 

•  Transport  a  casualty. 

•  Apply  life  saving  decision  skills  under  fire. 

Although  it  was  the  directive  of  the  ACOS  that  the  Warrior  Tasks  and  Battle  Drills  be 
tasked  to  all  Soldiers,  it  should  be  noted  that  as  official  policy,  training  on  Warrior  Tasks  and 
Battle  Drills  stops  upon  completion  of  IET,  and  there  is  no  requirement  that  the  tasks  be  carried 
over  to  unit  training  or  to  post-assignment  individual  training.  This  identifies  what  many  see  as  a 
disconnect  between  the  Warrior  Tasks  and  Battle  Drills  and  other  mandatory  training  requirements. 
This  difference  is  manifested  in  the  Common  Task  Test  (CTT),  which  is  the  only  doctrinally 
supported  Army- wide  statement  of  Soldier  performance  expectations.  The  2006  CTT  Task  List  is 
shown  in  Appendix  C  and  reflects  that  about  56%  of  potential  CTT  Tasks  share  commonality  with 
the  Warrior  Tasks  and  Battle  Drills.  This  is  actually  a  slight  decrease  from  the  2005  CTT  Task  List 
in  which  there  was  about  a  66%  overlap  of  tasks  (through  Skill  Level  2)  with  the  Warrior  list. 

The  Warrior  Tasks  and  Battle  Drills  is  a  crucial  source  for  the  Lessons  Learned  project 
because  of  its  prominence  and  Army  emphasis  in  IET.  However,  it  does  have  some  limitations 
for  this  research,  primarily  because  of  its  recency  and  limited  training  application.  Many 
incumbents  will  not  have  experienced  some  of  the  tasks  and  drills  included,  even  during 
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deployment.  In  addition,  Army  policy  is  that  the  list  is  subject  to  revision,  using  some  of  the 
same  criteria  that  we  used  in  the  Lessons  Learned  project.  Nonetheless,  it  is  a  fundamental 
source  for  the  current  work  and  was  given  significant  weight  in  the  final  taxonomy 
considerations.  All  of  the  Tasks  and  Battle  Drills  are  included  in  the  final  list. 

TRADOC-ARI  Common  Tasks  Survey 

In  2004,  TRADOC  and  the  ARI  Occupational  Analysis  Office  (OAO)  undertook  an 
Army-wide  survey  to  establish  new  training  requirements  for  common  tasks.  The  2004  work 
updated  a  similar  survey  conducted  in  1998  that  involved  an  inventory  of  about  900  tasks.  The 
2004  effort  sought  to  gather  information  and  confirm  common  task  requirements  in  light  of 
current  deployment  and  combat  operations.  The  updated  lists  were  based  in  part  on  Soldier 
observations  and  input  into  the  Center  for  Army  Lessons  Learned  (CALL).  The  purpose  of  the 
survey  was  to  support  updating  and  reorganization  of  most  officer  and  enlisted  professional 
development  programs,  including  a  revamping  of  the  NCO  Education  System  (NCOES).  Of 
importance  in  this  survey  was  the  emphasis  on  obtaining  participation  from  a  wider  respondent 
audience,  particularly  from  Reserve  Components. 

The  survey  was  targeted  towards  all  ranks  (enlisted,  warrant,  officer)  in  grades  E2 
through  06.  Each  group  surveyed  had  its  own  task  list.  The  survey  sought  to  confirm  the 
relevance  of  common  task  requirements  by  focusing  on  deployment  experiences,  asking:  “ While 
deployed  to  Iraq/Afghanistan  did/do  you  perform....  ”  Tasks  were  specific  and  organized  into 
task  categories.  Appendix  D  contains  a  summary  of  the  task  categories  for  Skill  Level  1  (E4)  and 
Skill  Level  2  (E5)  Soldiers. 

Of  all  of  the  sources  directly  accessed  for  the  Lessons  Learned  project,  this  source  is  the 
most  comprehensive  with  21  categories  and  276  tasks  defined  for  Skill  Level  1  and  12  categories 
and  142  tasks  identified  at  Skill  Level  2.  Ostensibly  at  least,  there  is  little,  if  any,  overlap  in  tasks 
between  the  two  Skill  Levels. 

The  survey  (which  was  voluntary  and  anonymous)  was  administered  via  the  web  and  was 
completed  in  December  2004  with  analysis  slated  to  continue  through  July  2005.  Although  we 
were  unable  to  obtain  results  to  incorporate  into  the  Lessons  Learned  project,  we  did  utilize  the 
task  listings  and  task  categories  in  our  consideration  of  source  input.  We  continue  to  track  the 
analysis  of  this  survey. 


The  Center  for  Army  Lessons  Learned  (CALL) 

The  Center  for  Army  Lessons  Learned  (CALL)  is  an  organization  of  TRADOC’ s 
Combined  Arms  Center  (CAC).  The  specific  mission  of  CALL  is  to  collect  and  analyze  data  and 
input  from  a  variety  of  sources  for  both  historical  and  operational  purposes.  CALL  distributes 
this  information  in  a  variety  of  formats,  including  a  website  (http://call.army.mil). 

Specific  to  this  research,  CALL  is  very  active  in  acquiring  and  posting  information  from 
both  OIF  and  OEF.  This  is  mostly  operational  information,  and  as  such,  is  in  a  restricted  portion 
of  the  website  that  is  not  publicly  accessible  (For  Official  Use  Only  [FOUO]).  No  classified 
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information  is  included  in  the  CALL  website.  Through  the  contracting  officer  representative 
(COR)  and  sponsor  authorization,  we  were  able  to  obtain  access  to  restricted  parts  of  the  website 
for  support  of  the  Lessons  Learned  project.  However,  some  information  on  the  site  is  read-only; 
it  cannot  be  copied,  printed,  downloaded,  or  saved. 

The  CALL  website  represents  an  unusually  rich  source  of  information  as  it  pertains  to 
lessons  learned  from  recent  deployments.  Foremost,  it  is  reflective  of  procedures  that  work  and 
have  been  battle  tested  under  operational  conditions.  It  is  the  most  relevant  source  for  the  project. 
It  is  also,  for  the  most  part,  timely,  reflecting  very  recent  feedback  from  field  forces.  However, 
there  are  inherent  challenges  with  the  CALL  information: 

•  Information  on  the  CALL  website  is  in  a  wide  variety  of  formats  and  presentations: 
Newsletters,  Smart  Cards,  Handbooks,  After  Action  Reports  (AAR),  Graphic 
Training  Aids  (GTA),  standing  operating  procedures  (SOPs),  briefing  slides, 
monographs,  videos,  and  training  support  materials.  Much  of  the  material  is  not 
“task-based”  and  takes  the  form  of  guidance,  information,  instruction,  or  “good-to- 
know”  material  for  tactics,  techniques,  and  procedures  (TTP).  Considerable  work  was 
required  to  extract  information  and  organize  it  into  a  format  suitable  to  serve  as 
source  material  for  this  research. 

•  Much  of  the  material  is  theater  (area)-specific  to  either  Iraq  or  Afghanistan  and  some 
is  even  area  of  operations  (AO)-specific  (e.g.,  3rd  Infantry  Division,  1st  Cavalry 
Division,  Baghdad,  Sunni  triangle,  Bagram).  Some  is  operation-specific,  for  example, 
lessons  learned  in  support  of  Phantom  Fury,  the  U.S.  Marine  Corps  assault  on  the 
strongholds  of  Fallujah.  As  a  result,  it  is  often  difficult  to  determine  how 
generalizable  the  information  is,  even  within  a  theater. 

•  The  criticality  of  the  task  or  subject  does  not  necessarily  translate  into  appropriate  test 
subject  matter.  For  a  Soldier  operating  in  the  mountains  in  Afghanistan,  knowing  the 
symptoms  of  high  altitude  sickness  may  be  a  life-saving  knowledge,  however  testing 
this  subject  Army-wide  would  not  likely  be  appropriate. 

•  Although  the  lessons  learned  information  from  the  CALL  resources  is  documented, 
much  of  the  documentation  is  neither  standardized  nor  available  in  standard  training 
sources  or  readily  available  references  for  study  and  referral. 

•  Some  of  the  information  is  mission,  situational,  or  time  sensitive.  Opposition  force 
tactics  in  both  theaters  have  changed  considerably  over  the  past  24  months  and  will 
likely  continue  to  evolve,  as  will  the  U.S.  and  Army  mission  and  supporting  tactics 
and  tasks. 

•  Change  is  constant  in  this  current-day  operating  environment.  New  information  is 
constantly  being  added  to  the  CALL  website.  Monitoring  and  adapting  to  lessons 
learned  updates  and  additions  is  a  persistent  requirement. 

•  Any  operational  material  extracted  from  the  website  was  treated  derivatively  as  For 
Official  Use  Only  (FOUO)  and  handled  accordingly,  even  if  the  information  was 
available  unrestricted  from  another  source.  None  of  the  information  in  this  report  falls 
into  that  restricted  category. 
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Project  personnel  with  site  access  spent  considerable  time  going  through  information  on 
the  website  and  organizing  that  information  into  a  list  that  approximates  a  task  list.  Much  of  the 
data  is  information  or  knowledge  that  is  necessary  for  Soldiers  operating  in  the  theater  and  no 
attempt  was  made  to  turn  this  into  task  statements.  Other  performance-based  requirements  are 
more  task-based,  but  the  CALL  source  mainly  provides  performance  requirements  and 
procedures  without  a  uniform  overlying  or  organizational  taxonomy.  Therefore,  project 
personnel  created  a  working  edition.  We  should  also  point  out  that  most  CALL  information  is 
not  Skill  Level-specific  or  otherwise  associated  with  Skill  Level,  and  while  no  attempt  was  made 
to  impose  a  Skill  Level  hierarchy,  the  organization  of  information  was  considered  appropriate  for 
the  target  test  audience.  A  list  of  the  categories,  topical  subject  areas,  and  “tasks”  is  presented  in 
Appendix  E.  The  information  included  in  Appendix  E,  while  originating  from  the  CALL 
website,  is  not  of  a  sensitive  nature. 

The  CALL  source  has  the  potential  to  be  the  most  meaningful  of  all  the  sources 
investigated  due  to  its  relevancy  and  timeliness.  However,  it  must  be  used  carefully,  particularly 
as  a  source  to  develop  test  items.  The  generalizability  of  the  material  must  be  established,  as 
must  its  doctrinal  accuracy  and  applicability.  By  way  of  demonstration  of  the  potential  pitfalls  is 
the  following  example  of  a  task  to  conduct  a  trauma  assessment  of  wounded/injured  person:  The 
“standard”  way  to  conduct,  sequentially,  trauma  assessment  is  to  remember  the  mnemonic 
“ABC”  (airway,  breathing,  circulation).  However,  because  of  the  nature  of  most  casualty  trauma 
in  Iraq,  the  operational  sequence  applied  there  is  “CAB”  (circulation,  airway,  breathing).  Both 
approaches  are  viable  and  doctrinally  correct  for  their  context  but  care  must  be  taken  when 
testing  such  a  task. 


Policy,  Command,  and  Authority 

The  final  source  is  a  somewhat  different  resource  than  the  previous  discussions.  It  is  used 
less  to  provide  content  and  more  as  a  means  of  interpreting,  prioritizing,  or  emphasizing  the  tasks 
or  other  content  gathered  elsewhere.  Army  policy  and  the  input  of  Army  leaders  and  other 
specialized  groups  are  also  critical  factors,  especially  in  the  current  operating  environment. 

Throughout  the  project  we  have  constantly  sought  Army  reviewer  input.  We  have  used 
the  ATP  AT  members  as  a  group  as  well  as  sub-groups  from  the  ATP  AT  for  more  targeted 
participation.  The  ATP  AT  is  particularly  important  in  this  regard  because  of  the  wide  range  of 
constituencies,  including  emphasis  on  the  Reserve  Components,  which  it  represents.  In  fact,  in 
developing  the  test  blueprint  for  the  PerformM21  Army- wide  common  core  exam,  the  ATP  AT 
served  as  the  entire  resource  including  vetting  (and  changing)  the  tasks  selected  for  test 
development. 

Another  source  of  input  occurs  when  the  PerformM21  project  is  briefed  to  high-level 
decision  makers.  They  often  provide  guidance  from  a  level  that  would  normally  not  be  available. 
For  example,  at  a  briefing  presented  to  TRADOC,  a  high  level  flag  officer  offered  that  “Detainee 
Operations”  was  an  extremely  high  level  priority  area  in  the  Army,  closely  followed  by  “Cultural 
Awareness”  and  “Sexual  Harassment.”  Moreover,  he  provided  his  rationale  for  his  support  of 
these  topics.  Such  information  not  only  makes  the  resulting  test  more  relevant  but  also  gives 
credibility  to  the  role  of  testing  as  a  means  of  enforcing  and  supporting  Army  policy  and 
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concerns.  Another  example  of  supporting  Army  policy  is  in  consideration  of  the  Warrior  Tasks 
and  Battle  Drills  (the  so-called  “39/9”)-  Because  we  were  aware  of  the  emphasis  of  this  source  in 
the  Army,  and  the  desire  to  extend  this  to  areas  beyond  IET,  we  gave  higher  priority  to  this  range 
of  tasks  when  actual  test  content  decisions  were  made. 

However,  an  equally  important  consideration  to  keep  in  mind  when  reacting  to 
individuals’  comments,  suggestions,  and  priorities  is  that  individuals  do  have  their  own  personal 
issues,  biases,  and  preferences,  which  may  not  reflect  overall  Army-wide  interests  or 
requirements.  The  potential  for  conflicting  priorities  exists,  and  the  program  cannot  thoughtlessly 
react  to  each  priority.  It  is  therefore  important  to  keep  in  place  a  system  of  checks  and  balances 
when  utilizing  this  category  of  sources  in  making  final  testing  decisions. 

Nonetheless,  to  ignore  the  realities  of  command  and  partisan  influence  is  to  peril  the 
credibility  and  acceptance  of  a  testing  program.  It  is  critical  that  individuals  and  groups  who 
review  testing  plans  have  access  to  all  the  data  results,  methodology,  and  other  information  that 
the  development  staff  used.  Then,  if  they  provide  reasoned,  directed  input  to  priorities  and 
emphasis,  it  should  be  factored  into  the  test  plan. 

Analysis  and  Consolidation  of  Source  Material 

Although  cursory  review  of  the  various  lists  used  in  the  Lessons  Learned  project  might 
lead  to  the  conclusion  that  they  are  disparate,  detailed  examination  reveals  considerable 
commonality  and  overlap.  Much  of  the  difference  in  the  source  lists  is  in  the  level  of  specificity 
or  abstraction  of  the  statements  and  in  the  way  that  requirements  are  aggregated.  (For  example, 
the  task  “Qualify  with  assigned  weapon”  is  actually  the  same  requirement  as  reflected  in  the 
three  tasks  “Engage  targets  with  an  M16  rifle,”  “Engage  targets  with  an  M9  pistol,”  and  “Engage 
targets  with  an  M4  carbine.)  Task  lists  and  task  descriptions  always  present  challenges.  Critics  of 
the  proliferation  of  lists  ask  (with  some  justification)  why  a  single  list  cannot  be  achieved.  In 
fact,  however,  task  lists  are  working  documents,  meant  to  be  used  as  tools  for  a  specific  purpose 
rather  than  end  products  that  exist  for  their  own  sake. 

Analysis  of  the  sources  used  in  this  work  reveals  that  each  of  them  was  designed  for  a 
particular  use.  As  such,  direct  application  in  another  context  is  not  entirely  suitable  as  noted 
below: 

•  The  Warrior  Tasks  and  Battle  Drills  were  designed  to  fit  a  training  need  for  a  very 
specific  group:  Soldiers  coming  out  of  IET  who  might  be  deployed  directly  to  a 
combat  theater.  The  developmental  history  of  Warrior  Tasks  and  Battle  Drills  shows 
a  potentially  much  larger  criticality  domain  with  final  structure  being  determined 
primarily  on  implementation  exigencies.  As  such,  in  its  current  form,  it  only 
represents  a  small  slice  of  the  probable  and  intended  zone  of  performance. 

Nonetheless,  this  source  has  high  current  visibility,  has  well-defined  content,  and  is 
gaining  acceptance  and  recognition  in  other  Army-wide  applications,  principally  the 
Common  Task  Test. 

•  The  TRADOC-ARI  survey  task  list  was  designed  as  a  tool  to  support  training 
emphasis  decisions.  It  is  deliberately  comprehensive  and  specific.  Ideally  structured 
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for  its  use  in  a  survey,  it  is  too  lengthy  and  cumbersome  to  use  in  its  raw  form.  Post¬ 
survey  analysis  should  winnow  the  list  to  a  more  manageable  number  and  order  of 
tasks.  However,  its  comprehensiveness  makes  it  a  good  standard  to  compare  with 
other  lists. 

•  The  CALL  website  is  a  unique  source.  It  is  designed  as  a  repository  of  materials  used 
by  operational  units.  It  does  not  purport  to  be  a  doctrinal  source  of  performance 
requirements  and  is  not  concerned  with  tasks  per  se.  It  is  not  comprehensive  and, 
indeed,  may  not  even  be  representative  of  the  overall  deployment  requirements.  It  is  a 
very  raw  source  and,  in  its  current  form,  largely  unprocessed.  However,  it  is  the  most 
germane  to  the  focus  of  the  Lessons  Learned  project  because  it  is  the  only  source  that 
is  generated  directly  from  the  field  and  deployment  experiences.  Used  judiciously,  it 
provides  a  rich  contribution  to  the  project,  particularly  in  the  phase  when  actual  test 
items  are  developed.  It  is  also  a  valuable  source  for  generating  new  tasks  and 
categories  of  tasks. 

•  The  Army- wide  prototype  test  developed  as  part  of  PerformM21  is  characterized  by 
the  fact  that  it  was  the  sole  source  that  went  beyond  operational  requirements  and 
explored  knowledges  in  areas  of  history,  leadership.  Army  Values,  and  customs,  and 
identified  training  knowledges.  It,  too,  filled  a  specific  requirement  of  reacting  to  an 
expressed  need  of  the  Army  leadership. 

The  ultimate  goal  of  reviewing  the  previously  described  sources  was  to  develop  a 
comprehensive  taxonomy  of  tasks  and  knowledges  to  serve  as  the  basis  for  a  job  analysis  survey. 
This  taxonomy  was  produced  by  iteratively  reviewing,  comparing,  contrasting,  and  synthesizing 
the  sources  previously  discussed.  Development  of  the  list  involved  the  following  considerations: 

•  The  taxonomy  would  clearly  distinguish  between  skills  (tasks)  and  knowledges. 

•  Although  having  a  lessons-learned  focus,  the  taxonomy  would  be  all-inclusive,  not 
just  focusing  on  deployment-oriented  requirements. 

•  The  tasks  and  knowledges  needed  to  be  abstracted  to  a  level  to  be  manageable  and 
not  present  an  onerous  requirement  to  reviewers  and  survey  respondents.  Moreover, 
the  primary  use  of  the  taxonomy  was  to  support  a  test  blueprint.  As  such,  we 
deliberately  avoided  over-defining  the  tasks  to  facilitate  long-term  use  of  the 
instrument. 

•  In  constructing  performance  or  knowledge  statements,  the  focus,  where  appropriate, 
was  on  identifying  competencies  rather  than  citing  specific  equipment  or  procedural 
parameters. 

•  Taxonomy  categories  (major  classifications  of  tasks  and  knowledges)  had  to  be  a 
manageable  number  and  have  meaningful  descriptors.  Reviewers  and  raters  would 
need  to  be  able  to  understand  and  rate  or  rank  at  the  category  level.  The  sizes  of  the 
categories  (numbers  of  tasks  and  knowledges)  needed  to  be  roughly  equal. 

Project  staff  synthesized  the  various  sources  into  a  preliminary  list.  Refinements  were 
conducted  by  collaboration  with  subject  experts  and  job  taxonomy  experts  until  a  final  list  was 
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determined.  The  result  (Appendix  F)  was  99  skill  task  areas  in  9  categories  and  23  knowledge 
areas  in  4  categories. 


Summary 

No  single  task  or  performance  requirements  list  is  likely  to  meet  all  the  diverse  needs  on 
uses  for  which  such  lists  are  normally  employed.  Moreover,  no  single  list  is  going  to  be  acceptable 
to  all  Army  reviewers  and  critics,  especially  if  those  reviewers  were  not  involved  in  the  process. 
We  recognize  that  there  are  many  divergent  priorities  deserving  emphasis  and  complete  agreement 
on  outcomes  is  not  a  reasonable  expectation.  Nonetheless,  the  process,  if  not  the  outcome,  should 
be  defensible  and  acceptable.  Such  was  the  approach  used  in  identifying  the  domain  for  the 
Lessons  Learned  project.  By  accessing  a  number  of  recent  and  relatable  sources,  analyzing  them, 
and  integrating  the  results,  we  have  worked  to  create  a  list  that  not  only  represents  a  reasonable 
performance  domain  but  that  also  supports  the  target  aims  of  the  project:  emphasizing  skills  and 
knowledges  that  support  deployment  operations  in  the  current  war  climate. 

At  the  same  time,  this  approach  to  incorporating  lessons  learned  material  provokes 
caution  and  attention  in  the  way  that  it  is  applied.  Procedures  derived  from  lessons  learned  can 
be  idiosyncratic  and  temporary,  often  to  be  replaced  by  fresh,  more  contemporary  procedures. 
The  current  operating  environment,  with  ever-changing  enemy  composition  and  tactics,  is 
particularly  difficult  to  operationally  define  and  stabilize.  Moreover,  the  paradigm  for  Army 
training  has  shifted  to  one  where  doctrine  is  being  defined  and  determined  by  forces  in  the  field, 
rather  than  by  traditional  institutions  and  schoolhouse  organizations.  Facilitated  by  the  Internet 
and  special  interest  websites,  word  is  spreading  faster  to  interested  users  before  it  can  be 
evaluated  and  documented  by  specialized  doctrine  reviewers.  To  remain  consistent  in  light  of 
shifting  inputs,  we  have  concentrated  on  identifying  task  or  knowledge  areas  rather  than  specific 
techniques  and  procedures.  In  this  approach,  the  generated  Lessons  Learned  list  should  remain 
viable  over  a  longer  term. 

The  list  of  performance  requirements  in  Appendix  F  is  an  interim  product,  designed  to  be 
modified  based  on  survey  results  and  other  review  input.  It  reflects  the  best  approach  to 
organizing  information  obtained  from  very  credible  and  functional  sources.  This  list  should  serve 
as  a  solid  foundation  on  which  to  base  test  blueprint  design  activities. 
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CHAPTER  3:  SURVEY  AND  BLUEPRINT  DEVELOPMENT 


Design  Goals 

Our  ultimate  goal  was  to  construct  a  prototype  survey  that  would  support  development  of 
a  common  core  test  blueprint  with  several  desirable  features: 

•  Accurate  reflection  of  Soldier  job  demands,  particularly  as  they  regard  deployment 
requirements. 

•  Test  blueprint  categories  that  are  maximally  useful  for  test  development. 

•  Test  blueprint  categories  that  are  maximally  useful  for  Soldier  preparation  and 
feedback. 

The  fundamental  goal  of  an  effective  competency  assessment  program  is  the  act  of 
teaching  to  the  test  to  result  in  improved  job  performance.  Moreover,  this  feature  should  be 
evident  to  stakeholders.  That  is.  Soldiers  and  their  leaders  should  be  able  to  look  at  a  test  or  a  test 
blueprint  and  perceive  that  it  corresponds  to  field  requirements,  even  though  it  is  unlikely  to 
strictly  match  the  requirements  of  a  given  Soldier’s  current  assignment.  Accordingly,  test  content 
must  reflect  performance  as  it  is  defined  by  Soldiers  in  the  field.  The  purpose  of  using  a  survey 
to  collect  input  is  to  ensure  that  test  specifications  are  not  unduly  influenced  by  the  experiences 
of  a  relatively  small  number  of  Soldiers.  In  addition  to  being  relevant,  test  content  should  be 
specified  broadly  enough  to  prevent  Soldiers  from  preparing  to  be  tested  on  a  relatively  small 
number  of  specific  tasks.  While  such  a  test  can  be  useful  for  supporting  training  activities,  it  is 
not  likely  to  be  reflective  of  a  broad  enough  sampling  of  the  job  to  conclude  that  high  scorers  are 
fully  qualified  to  perform  their  jobs.  This  was  a  criticism  of  the  old  Skill  Qualification  Tests 
(SQTs)  and  is  a  feature  of  the  Army’s  current  Common  Task  Test.  Thus,  we  opted  for 
developing  a  prototype  test  blueprint  that  specifies  categories  of  content  rather  than  very  specific 
tasks  or  knowledges. 

Using  broad  categories  of  content  also  facilitates  test  development.  The  idea  is  that  test 
developers  can  develop  “banks”  of  items  for  each  blueprint  category.  When  constructing  a  new 
test  form,  the  appropriate  number  of  items  for  each  category  is  sampled  from  the  item  bank.  This 
allows  quite  a  bit  of  latitude  in  item  selection,  making  development  of  multiple  forms  with 
different  items  but  comparable  content  feasible. 

A  test  blueprint  based  on  categories  of  content  helps  both  test  developers  and  examinees 
because  the  blueprint  will  not  change  dramatically  from  one  test  cycle  to  the  next.  Barring 
fundamental  changes  to  the  job,  for  example,  a  certain  percentage  of  the  test  will  always  cover 
first  aid  related  tasks  and  knowledges  and  this  percentage  is  not  likely  to  vary  much  from  year  to 
year.  Certain  tasks  and  knowledges  will  change  over  time,  but  this  will  not  be  evidenced  in 
constant  changes  to  the  test  blueprint,  but  rather  to  edits  to  the  item  bank  to  maintain  its 
currency.  This  stability  is  in  the  best  interests  of  test  developers,  trainers,  and  examinees. 

The  Army  is  interested  in  an  assessment  program  that  will  provide  useful  performance 
feedback  to  Soldiers.  This  usually  means  calculating  subtest  (as  well  as  total)  scores  on  the  test 
and  reporting  these  back  to  examinees.  For  subscores  to  be  informative,  they  should  be  based  on 
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test  items  with  related  content  (e.g.,  not  mixing  first  aid  and  weapons  items),  and  there  should  be 
enough  items  on  the  subtest  to  result  in  reliable  scores.  A  subtest  score  based  on  5  items  sampled 
from  the  item  bank,  for  example,  is  not  going  to  be  very  reliable  whereas  a  subtest  score  based 
on  15  or  20  items  would  likely  be  sufficiently  reliable  to  be  useful  feedback. 

A  Working  Process  Model 

The  Army  already  has  an  occupational  analysis  program  designed  to  identify  training 
requirements,  as  described  in  TRADOC  Regulation  350-70.  Given  the  aforementioned  goals  for 
test  design,  however,  occupational  analysis  surveys  developed  under  that  program  do  not  provide 
the  required  information  because  their  focus  is  on  identifying  Soldier  training  needs.  Rather  than 
design  a  completely  new  occupational  analysis  program  that  might  be  able  to  address  the 
information  requirements  for  both  training  and  testing,  the  PerformM21  test  program  design 
recommendations  suggest  a  model  similar  in  concept  to  that  used  by  the  Air  Force  (see 
www.icodap.or&).  Specifically,  the  Air  Force  uses  detailed  task  analysis  surveys  that  are 
designed  to  identify  training  needs  to  and  identify  the  critical  tasks  that  will  be  considered  for 
job-specific  testing.  Additional  data  to  support  development  of  a  test  blueprint  are  collected  via  a 
separate,  smaller-scale  “Test  Importance  Survey”  that  is  administered  only  to  supervisors.  We 
envision  the  Army  adopting  a  similar  model,  in  which  training-focused  occupational  analysis 
results  are  used  to  provide  a  starting  point  for  identifying  the  content  that  should  be  included  in 
smaller-scale  test  emphasis  surveys  (Knapp  &  Campbell,  2006). 

Development  of  Blueprint/Survey  Content  Categories 

A  key  question  in  determining  survey  content  is  to  decide  on  the  nature  of  the  job  descriptors 
that  would  best  serve  test  design  purposes.  We  have  previously  discussed  the  advantages  of  using  a 
knowledge-based  test  blueprint  as  a  foundation  for  a  multiple-choice  exam  (Knapp  &  Campbell, 
2006).  Depending  upon  the  nature  of  the  job  requirements,  however,  other  choices  might  be  more 
sensible.  For  a  core  Army  examination,  we  believe  that  a  primarily  task-based  focus,  albeit,  using 
task  categories  and  less  detailed  constituent  tasks  than  the  Army  uses  for  training  applications,  would 
be  the  most  appropriate  choice1.  This  is  because  our  review  of  the  PerformM21  common  core  test 
blueprint  suggests  that  there  is  not  a  lot  to  be  gained  by  this  “conversion.”  Take  fast  aid  tasks  as  an 
example.  Most  Skill  Level  1  Soldiers  are  expected  to  memorize  what  they  are  supposed  to  do  rather 
than  understand  why  they  are  doing  a  first  aid  task  in  a  particular  way.  In  such  a  case,  asking  about 
procedures  (as  would  be  easy  to  do  from  a  task  category  based  blueprint)  would  be  as  easy,  or  easier, 
than  trying  to  infer  an  underlying  knowledge  base. 

As  discussed  in  Chapter  2,  project  staff  developed  task  statements  primarily  from  the  latest 
Army  training-oriented  common  task  survey.  There  were,  however,  some  knowledge  areas  that  were 
viewed  as  important,  but  which  would  not  naturally  surface  on  a  test  based  on  the  common  tasks.  As 
described  in  Chapter  2,  these  knowledge  areas  were  identified  primarily  by  our  SMEs. 

It  was  challenging  to  organize  task  and  knowledge  statements  into  completely 
satisfying  categories.  Creating  categories  that  were  homogeneous  in  content  resulted  in  some 


1  Note  that  the  type  of  task  we  discuss  here  does  not  meet  TRADOC’ s  definition  of  a  “task”  as  it  applies  to  their 
occupational  analysis  protocols  (see  TRADOC  Regulation  350-70). 
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categories  that  were  very  large  and  others  that  were  very  small.  As  such,  they  were  not  suitable 
for  our  needs.  We  worked  with  SMEs  to  develop  categories  that  were  more  even  in  size.  This 
still  resulted  in  a  fairly  large  number  of  categories  (13),  so  we  organized  those  into  five  “meta¬ 
categories.”  The  meta-categories  are  useful  for  getting  a  succinct  “big  picture”  of  what  might  be 
covered  on  a  test  and  for  obtaining  holistic  judgments  about  test  coverage  (see  related  discussion 
later  in  this  chapter).  Table  1  shows  the  nine  task  and  four  knowledge  categories  sorted  into  five 
meta-categories.  Although  it  is  likely  that  improvements  can  be  made  to  this  categorization 
scheme,  we  believe  this  set  of  categories  brings  the  Army  close  to  something  that  would  be 
suitable  for  an  operational  common  core  test  program. 


Table  1.  Survey  Categories 


Survey  Categories 

Number  of 

Tasks/Knowledge  Areas 

Self  Protection  and  Defense 

35 

First  Aid,  Preventive  Medicine,  and  MEDEVAC 

18 

NBC  Protection  and  Decontamination 

8 

Threat  Identification  and  Reaction 

9 

Individual  Weapons,  Communications,  and  Navigation 

22 

Navigation,  Communication,  Call  for  Fires 

8 

Weapons,  Grenades,  Mines,  and  Demolitions 

14 

Tactical  Operations 

34 

Vehicle  and  Convoy  Tactical  Operations 

9 

Force  Protection,  Crowd  Control,  Security,  and  Intelligence,  Detainee  Operations 

13 

Urban  Opertions,  Camouflage,  Fighting  Positions 

12 

Supervision 

15 

Supervise,  Lead,  and  Train  Others 

8 

Leadership  Procedures  and  Principles 

7 

Army  Rules  and  History 

16 

Rules,  Regulations,  Laws  of  Conduct 

6 

Army  Policies  and  Practices 

6 

Army  History,  Customs,  Values 

4 

Survey  Strategy 
Choice  of  Respondents 

Particularly  at  the  E4  paygrade,  we  can  expect  incumbents  to  provide  reasonably  accurate 
information  about  what  they  do  and  how  often  they  do  it.  We  would  not  expect  them  to 
accurately  judge  what  other  E4  Soldiers  do  (especially  when  considering  requirements  across  the 
Army),  nor  would  we  expect  them  to  be  the  best  judges  of  the  knowledges,  skills,  and  other 
attributes  (KSAs)  required  for  effective  job  performance.  Moreover,  we  should  carefully 
consider  the  limited  perspective  of  most  E4  Soldiers  when  asking  them  about  the  importance  or 
criticality  of  the  various  things  they  do.  In  contrast,  supervisors  may  be  viewed  as  a  preferred 
source  for  task  importance  and  KSA  information.  This  is  particularly  true  for  low  skill 
occupations  (as  opposed  to  those  in  which  incumbents  are  highly  educated).  Thus,  we  developed 
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an  automated  survey  with  branching  characteristics  that  made  it  suitable  for  two  types  of 
respondents  to  provide  input — incumbents  (E4  Soldiers)  and  supervisors  (E5-E8  NCOs). 

Choice  of  Rating  Scales 

Our  goal  was  to  determine  what  rating  scale(s)  would  yield  the  most  reliable, 
comprehensive,  and  complete  data  maximally  suitable  for  purposes  of  test  design.  We  presumed 
the  goal  of  complete  data  would  be  facilitated  by  using  the  least  number  of  rating  scales  possible 
(thus  reducing  the  time  required  to  complete  the  survey),  so  we  kept  the  number  of  scales  to  a 
minimum. 

Standard  Practices 

Task  inventories  typically  ask  one  or  more  of  the  following  questions  (e.g.,  Raymond, 

2001): 

•  How  important  or  significant  is  each  task  to  your  job? 

•  What  are  the  consequences  of  error  (task  criticality)? 

•  How  much  time  do  you  spend  on  each  task? 

•  How  frequently  do  you  perform  each  task? 

•  How  difficult  is  it  to  perform  each  task? 

We  collected  examples  of  scales  (questions  and  response  options)  from  the  literature,  the  other 
services,  and  our  previous  work  with  similar  surveys. 

The  selection  of  rating  scales  should  be  determined  based  on  what  information  is  required 
in  a  given  situation.  Moreover,  consideration  should  be  given  to  whether  different  scales  provide 
sufficiently  unique  information  to  warrant  inclusion.  For  example,  research  suggests  that  time 
spent  and  frequency  ratings  tend  to  be  fairly  highly  correlated  regardless  of  job  type  (Friedman, 
1990).  The  correlation  between  importance  and  criticality  (consequences  of  errors)  is  usually 
also  fairly  high  (Sanchez  &  Fraser,  1992),  however,  this  can  vary  across  job  types.  Often,  if  is 
necessary  to  obtain  an  overall  indicator  that  incorporates  several  judgments  about  the  task  (e.g., 
performance  frequency  and  importance).  It  is  possible  to  get  such  an  overall  indicator  either  by 
statistically  combining  ratings  on  multiple  scales  or  by  asking  raters  to  consider  multiple  factors 
(e.g.,  time  spent,  criticality)  when  making  a  single  rating  (Cornelius  &  Lyness,  1980). 

Special  Considerations 

Some  of  the  relatively  unique  issues  associated  with  collecting  job  analysis  ratings  from 
military  personnel  are  described  below: 

•  Job  requirements  between  garrison  and  combat  settings  may  differ,  with  the 
differences  being  more  or  less  pronounced  depending  on  MOS  (relates  to  the  context 
in  which  respondents  are  asked  to  make  their  ratings). 
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•  Reserve  Component  Soldiers  may  not  have  recent  experience  working  in  their  MOS 
in  a  full-time  capacity,  so  absolute  frequency  scales  would  not  be  meaningful  for  part- 
time  respondents. 

•  Soldiers  perform  a  mix  of  Army-wide  and  MOS-specific  tasks.  Use  of  relative  scales 
(often  preferred  with  the  expectation  they  will  show  more  variability  than  absolute 
scales)  might  reduce  the  comparability  of  ratings  on  Army-wide  tasks  collected  from 
Soldiers  in  different  MOS. 

The  Selected  Rating  Scales 

Although  this  might  not  be  true  at  higher  paygrades,  we  would  expect  E4  Soldiers 
(incumbents)  to  be  better  able  to  describe  what  they  do  and  how  often  they  do  it  than  to  make 
other  judgments  about  their  job  tasks  (e.g.,  importance  to  mission  or  recommended  test 
emphasis).  We  decided  to  have  both  incumbents  and  supervisors  rate  the  tasks  on  a  “time  spent” 
scale.  We  considered  several  options  for  this  scale,  looking  for  something  that  would  have  a 
consistent  absolute  and  interpretative  meaning  whether  the  incumbent  is  a  full-time  or  part-time 
Soldier.  We  selected  time  spent  rather  than  frequency  for  this  reason  and  because  we  could  use 
anchors  with  interval  scale  properties.  The  specific  scale  we  used  is  shown  in  Figure  2. 

In  an  operational  survey,  we  are  leaning  toward  recommending  collection  of  only  time 
spent  or  frequency  data  from  E4  Soldiers,  and  then  only  if  it  serves  as  a  reality  check  for  a  test 
blueprint  that  would  be  based  primarily  on  test  emphasis  ratings  from  supervisors.  For  purposes 
of  the  Lessons  Learned  project  survey,  however,  we  collected  data  on  both  time  spent  and  test 
emphasis  from  both  incumbents  and  supervisors  so  that  we  could  examine  the  correlation 
between  the  two  scales  and  compare  the  ratings  collected  from  incumbents  and  supervisors. 

Just  as  the  services  often  use  a  “training  emphasis”  rating  rather  than  (or  in  addition  to) 
the  other  questions  that  typically  appear  on  task  inventories,  we  used  a  “test  emphasis”  rating  to 
collect  information  on  both  the  tasks  and  knowledges  listed  on  the  survey  (see  Figure  2).  This  is 
similar  to  the  “test  importance”  rating  collected  by  the  Air  Force,  although  the  instruction  set  and 
rating  scales  are  different.  We  considered  asking  for  ratings  of  both  task  importance  and  test 
emphasis,  but  expected  the  two  judgments  to  be  highly  correlated  and  we  wanted  to  keep  survey 
administration  time  requirements  to  a  minimum.  Note  that  this  single  test  emphasis  rating  uses 
the  strategy  described  previously  of  asking  respondents  to  consider  several  factors,  then 
combining  these  into  a  single  judgment. 

We  used  5-point  scales  (with  a  sixth  “not  applicable”  option)  primarily  in  an  effort  to 
simplify  the  rating  activity  for  respondents.  The  rating  scale  anchors  were  worded  to  be  relative 
judgments  (i.e.,  comparing  to  other  tasks  or  knowledges)  rather  than  absolute  judgments  to 
encourage  respondents  to  use  the  entire  5-point  scale.  We  phrased  the  rating  scale  questions  and 
scale  anchors  in  the  most  equivalent  manner  possible  to  permit  comparison  for  comparable 
questions  across  different  respondent  types  (e.g.,  incumbents  and  supervisors). 
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•  How  much  time  do  you  [“Specialists/Corporals,”  for  supervisors]  spend  performing  this  task 
compared  to  other  tasks  [knowledges].  Consider  time  spent  during  daily  work,  training,  and 
deployment  activities. 

0.  None 

1 .  Very  small  amount 

2.  Below  average 

3.  About  average 

4.  Slightly  above  average 

5.  Above  average 

•  How  much  should  this  task  [or  knowledge]  be  emphasized  on  a  test  for  Specialists/Corporals 
competing  for  promotion  to  Sergeant?  When  considering  your  rating,  think  about  these  factors: 

-  Time  spent  on  task 

-  Seriousness  of  errors 

-  Importance  to  unit  mission 

-  Variability  in  ability  to  perform 

0.  Do  not  include  on  test 

1 .  Much  less  emphasis  on  the  test  compared  to  other  tasks  [knowledges] 

2.  Less  emphasis  on  the  test  compared  to  other  tasks  [knowledges] 

3.  About  the  same  emphasis  on  the  test  compared  to  other  tasks  [knowledges] 

4.  More  emphasis  on  the  test  compared  to  other  tasks  [knowledges] 

5.  Much  more  emphasis  on  the  test  compared  to  other  tasks  [knowledges] 


Figure  2.  Survey  rating  scales. 

Occupational  analysis  survey  designers  will  sometimes  require  respondents  to  rate  all 
items  (e.g.,  tasks)  on  one  scale  (e.g.,  time  spent)  before  rating  them  on  a  second  or  third  scale 
(e.g.,  importance).  Although  this  might  help  raters  focus  on  what  judgment  they  are  making,  it 
makes  the  survey  more  time-consuming  to  complete.  Thus,  our  survey  was  designed  so  that 
respondents  rated  each  task  on  time  spent  and  test  emphasis  before  going  to  the  next  task.  This  is 
different  than  the  Army’s  training  surveys  and  may  not  be  optimal  from  a  research  perspective, 
but  it  made  the  rating  activity  easier  and  quicker  for  respondents. 

Holistic  Weighting  Exercise 

We  wanted  to  compare  two  strategies  for  determining  how  much  emphasis  to  give 
various  categories  of  test  content.  One  strategy  would  be  based  on  combining  test  emphasis 
ratings  made  on  individual  task  and  knowledge  statements.  The  other  strategy  would  require 
supervisors  to  make  a  direct  judgment  regarding  the  weight  that  should  be  given  to  each  test 
category.  We  have  successfully  used  this  method  for  developing  civilian  certification  test 
blueprints,  and  implement  it  by  having  survey  respondents  allocate  100  points  across  all  possible 
content  categories.  As  previously  discussed,  we  thought  that  it  would  be  too  demanding  to  ask 
survey  respondents  to  allocate  points  across  1 3  categories,  so  we  had  them  do  this  exercise  only 
for  the  5  meta-categories. 
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Background  and  Reaction  to  Survey  Questions 

We  developed  questions  for  the  survey  to  collect  information  from  respondents  that 
would  help  to  evaluate  the  representativeness  of  the  survey  sample  and  allow  for  analysis  of 
results  by  subgroups  of  interest  (e.g.,  respondents  who  have  been  deployed  compared  to  those 
who  have  not). 

We  gave  considerable  thought  to  the  deployment  experience  questions  as  they  have  the 
potential  to  be  informative  for  thinking  about  integrating  lessons  learned  during  recent 
deployments  into  the  resulting  test  blueprint.  It  is  also  the  case  that  being  “deployed”  can  mean  a 
number  of  things  across  different  contexts.  We  looked  at  how  other  surveys  handle  questions  about 
deployments  and  settled  on  an  approach  we  devised  to  have  respondents  indicate  the  types  of  roles 
they  filled  during  their  deployment.  These  eight  “roles”  characterize  different  types  of  activities 
that  Soldiers  are  required  to  perform  that  fall  outside  of  their  normal  MOS  responsibilities.  It  is  our 
understanding  that  many  Soldiers  work  outside  their  MOS  while  deployed,  doing  activities  that  are 
not  necessarily  associated  with  any  other  MOS.  This  information  would  be  of  general  interest  and 
could  possibly  factor  into  test  blueprint  decisions.  These  eight  roles  are  detailed  in  a  later  section. 

Finally,  we  developed  questions  to  help  evaluate  how  respondents  felt  about  the  survey. 
Questions  concerned  how  easy  or  difficult  it  was  to  complete  various  parts  of  the  survey. 

Survey  Construction2 

Recognizing  that  Soldiers  and  NCOs  are  tasked  with  completing  surveys  all  the  time,  we 
endeavored  to  make  this  survey  as  appealing  and  non-burdensome  as  possible.  The  survey  was 
designed  to  be  an  automated,  Internet-based  survey.  Additionally,  the  following  qualities  were 
desired: 

•  Pause  feature  -  so  respondents  do  not  have  to  complete  the  entire  survey  in  one  setting. 

•  Branching  feature  -  so  incumbents  and  supervisors  were  branched  to  different 
versions  of  the  survey  (as  previously  noted),  and  so  that  Soldiers  below  paygrade  E4 
and  above  paygrade  E8  were  exited  from  the  survey. 

•  Easy  to  navigate. 

•  Logical  steps  for  some  items  -  for  instance,  so  that  applied  weights  could  not  exceed  100. 

•  Easy  for  researchers  to  access  the  data. 

We  constructed  the  survey  in-house  using  PHP  scripting  software.3  We  instructed 
respondents  to  create  their  own  user  name  and  password,  and  strongly  encouraged  them  to  select 
something  easy  to  remember  in  case  they  wanted  to  pause  the  survey  and  come  back  to  it  later. 

The  survey  was  hosted  on  the  HumRRO  server.  Because  respondents  were  not  known 
ahead  of  time,  we  were  not  able  to  restrict  access  to  the  site.  For  this  research  we  do  not  consider 
this  as  much  a  security  issue  as  a  data  integrity  issue.  None  of  the  content  on  the  survey  was 
FOUO  or  required  any  clearance.  Nor  did  we  collect  or  store  any  personal  identifiers  (i.e.,  social 

2  Shelly  West  (HumRRO)  was  critical  in  the  automation  of  this  survey,  both  for  her  insight  and  technical  capabilities. 

3  PHP  originally  stood  for  Personal  Home  Page,  but  now  has  come  to  mean  PHP:  Hypertext  Preprocessor. 
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security  numbers,  names).  Obviously  any  operational  survey  process  that  includes  sensitive 
information  -  whether  in  survey  content  or  respondent  data  -  should  restrict  access  to  the  site. 
The  data  integrity  concerns  result  from  having  no  way  to  prevent  respondents  from  completing 
the  survey  multiple  times,  or  from  non-Soldiers  completing  the  survey. 

Survey  Administration 

Our  goal  was  to  have  150  Soldiers  and  150  NCOs,  representing  both  Active  and  Reserve 
Components,  complete  this  survey.  We  did  not  have  a  list  of  potential  respondents,  however,  nor 
were  any  units  specifically  tasked  to  complete  the  survey.  Instead,  we  relied  on  the  ATPAT  to 
help  us  locate  respondents.  We  asked  them  to  forward  an  “advertisement”  email  down  their 
chain  of  command,  which  some  did.  We  also  posted  a  notice  on  a  commonly  accessed  NCO 
website  (www.NCOTeam.org).  By  far  the  most  successful  tool  for  gaining  respondents, 
however,  was  recruiting  participants  from  the  later  FY05  PerformM21  Phase  III  MOS  data 
collections.  We  asked  data  collectors  to  include  this  as  an  activity  to  be  completed  after  the  pilot 
tests. 


Click  here  to  see  a  complete  list  of  tasks  to  review  before  you  begin  Once  you  start  making  your  ratings,  click  on  "Ratine  Sc  ale 11 
to  see  the  full  text  of  the  rating  scales  again. 

A.  First  Aid,  Preventive  Medicine,  and  Medical  Evacuation 
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Figure  3.  Screen  shot  of  Army  Job  Analysis  Survey. 


Respondents  were  instructed  to  navigate  to  the  website,  review  the  project  briefing,  read 
and  accept  the  Privacy  Act  Statement,  complete  some  background  questions  (e.g.,  time  in  service, 
age,  gender),  and  then  complete  the  survey.  Figure  3  provides  a  screen  shot  of  the  survey. 

Data  Processing  and  Sample  Description 

It  originally  appeared  that  we  had  160  respondents  to  the  survey,  but  further  investigation 
revealed  that  57  (36%)  of  those  did  not  actually  answer  any  questions.  Anecdotal  evidence 
suggests  that  some  respondents  may  have  confused  this  survey  with  a  training  needs  analysis 
survey  being  conducted  around  the  same  time.  After  logging  in  successfully,  they  may  have 
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mistakenly  believed  they  already  completed  the  survey,  and  then  immediately  logged  out. 
Further,  another  eight  respondents  only  completed  the  background  information.  Therefore,  these 
65  respondents  were  dropped  from  the  data  set.  This  left  us  with  66  incumbents  and  29 
supervisors.  From  here  we  dropped  another  13  incumbents  and  seven  supervisors  for  too  many 
missing  responses  or  patterned  responding  (i.e.,  selecting  1  for  everything).  This  left  us  with  a 
sample  of  53  incumbents  and  22  supervisors,  47%  of  our  original  sample. 

Table  2  describes  the  incumbent  and  supervisor  survey  samples.  We  were  fairly 
successful  obtaining  input  from  Active  duty  and  U.S.  Army  Reserve  (USAR)  respondents,  but 
less  so  with  the  Army  National  Guard  (ARNG).  The  majority  (57%)  of  the  incumbent  sample 
had  been  deployed  within  the  previous  36  months  and  almost  all  members  of  the  supervisor 
sample  (96%)  had  been  deployed.  The  supervisor  sample  included  NCOs  in  E5  through  E8 
paygrades.  The  53  incumbents  came  from  16  different  MOS,  with  the  largest  percentage  (28%) 
from  31B  (Military  Police).  The  22  supervisors  came  from  17  different  MOS. 

Table  2.  Survey  Sample  Description 


Incumbents _ Supervisors 


Sample  Size 

53 

22 

Component 

Active 

25 

47% 

16 

73% 

ARNG 

4 

8% 

0 

USAR 

22 

42% 

6 

27% 

Missing 

3% 

Recently  Deployed 

30 

57% 

21 

96% 

Paygrade 

E5 

6 

27% 

E6 

8 

36% 

E7 

2 

9% 

E8 

6 

27% 

Gender 

Female 

7 

13% 

3 

14% 

Male 

46 

87% 

18 

86% 

Hispanic 

Yes 

10 

19% 

1 

5% 

No 

43 

81% 

21 

95% 

Race3 

Black 

8 

15% 

4 

18% 

White 

38 

72% 

17 

77% 

Other  -  Native  American,  Asian,  1 1 

21% 

0 

Pacific  Islander 


“Totals  exceed  100%  because  respondents  were  allowed  to  select  more  than  one  race  category. 
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Recall  that  we  asked  respondents  to  indicate  the  percentage  of  time  they  spent  on  eight 
types  of  duties  while  deployed.  We  thought  that  this  information  might  be  useful  for 
understanding  variation  we  might  see  in  responses  to  other  sections  of  the  survey,  although  the 
sample  sizes  were  too  small  to  analyze  the  data.  As  shown  in  Table  3,  most  incumbent 
respondents  who  had  recently  been  deployed  reported  some  involvement  in  direct  humanitarian 
assistance  and  force  protection.  Few  reported  involvement  in  intelligence  operations  or  major 
construction. 


Table  3.  Percent  Time  Spent  on  Selected  Deployment  Duties 


Rank 

Incumbent  Supervisor 

Freq  %  Freq  % 

Teach,  train,  instruct 

None 

16 

53.3 

9 

42.9 

Small 

6 

20.0 

3 

14.3 

Moderate 

6 

20.0 

6 

28.6 

Large 

2 

6.7 

3 

14.3 

Provide  direct  humanitarian  assistance 

None 

12 

40.0 

8 

38.1 

Small 

9 

30.0 

8 

38.1 

Moderate 

9 

30.0 

3 

14.3 

Large 

0 

2 

9.5 

Participate  in  search  and  destroy 

None 

17 

56.7 

13 

61.9 

Small 

9 

30.0 

3 

14.3 

Moderate 

4 

13.3 

4 

19.0 

Large 

0 

1 

4.8 

Perform  force  protection 

None 

11 

36.7 

3 

14.3 

Small 

6 

20.0 

4 

19.0 

Moderate 

7 

23.3 

11 

52.4 

Large 

6 

20.0 

3 

14.3 

Control  flow  and  movement 

None 

15 

50.0 

11 

52.4 

Small 

7 

23.3 

4 

19.0 

Moderate 

7 

23.3 

2 

9.5 

Large 

1 

3.3 

4 

19.0 

Provide  medical,  supply,  maintenance 

None 

12 

40.0 

12 

57.1 

Small 

8 

26.7 

3 

14.3 

Moderate 

4 

13.3 

1 

4.8 

Large 

6 

20.0 

5 

23.8 

Conduct  intelligence  operations 

None 

22 

73.3 

13 

61.9 

Small 

4 

13.3 

3 

14.3 

Moderate 

4 

13.3 

3 

14.3 

Large 

0 

2 

9.5 

Provide  major  construction 

None 

27 

90.0 

16 

76.2 

Small 

3 

10.0 

2 

9.5 

Large 

0 

3 

14.3 

20 


Note.  Incumbent  n  =  30,  supervisor  n  =  21.  Small  =  1-20%,  Moderate  =  21-80%,  Large  =  81-100%. 

Although  suitably  diverse,  the  survey  samples  are  too  small  to  be  considered 
representative  or  to  yield  reliable  results.  We  did  analyze  the  data,  however,  to  show  trends  and 
to  demonstrate  how  survey  data  can  be  used  to  construct  a  test  blueprint. 

Exploratory  Analyses 

Task  Statement  Ratings 

We  began  by  examining  the  task  ratings.  Appendix  G  (Table  G.l)  lists  the  mean  task 
time  spent  ratings  for  the  incumbent  and  supervisor  samples,  as  well  as  for  the  total  group.  Table 
G.2  lists  the  same  information  for  the  mean  task  test  emphasis  ratings.  There  was  one  task 
(Perform  first  aid  for  cold  injuries)  that  received  a  zero  rating  for  time  spent.  The  remaining 
mean  time  spent  ratings  hovered  mostly  in  the  middle  of  the  5-point  rating  scale  (i.e.,  most 
ratings  falling  between  2.50  and  3.50,  with  none  over  4.0).  This  was  the  case,  despite  the  fact 
that  the  survey  used  a  “relative”  rating  scale  that  should  theoretically  yield  more  spread  across 
the  scale.  Examination  of  the  test  emphasis  ratings  showed  a  similar  pattern,  although 
supervisors  showed  a  somewhat  greater  tendency  to  use  more  of  the  scale  at  the  higher  end,  with 
18  out  of  99  tasks  rated  higher  than  4.0. 


Table  4.  Category  Mean  Time  Spent  Ratings  by  Group 


Incumbent 

Supervisor 

Total 

M 

n 

SD 

M 

n 

SD 

M 

N 

SD 

First  Aid 

2.50 

53 

.86 

2.69 

22 

.71 

2.56 

75 

.82 

NBC  Protection  and  Decontamination 

2.48 

50 

.96 

2.39 

21 

.96 

2.46 

71 

.96 

Vehicle  and  Convoy  Operations 

2.84 

52 

.96 

3.25 

22 

1.07 

2.96 

74 

1.01 

Threat  ID  and  Reaction 

2.61 

53 

.94 

3.00 

22 

1.02 

2.72 

75 

.97 

Force  Protection 

2.46 

52 

.87 

2.79 

22 

.95 

2.56 

74 

.90 

Urban  Operations,  Camouflage 

2.38 

52 

.77 

2.66 

22 

.92 

2.46 

74 

.82 

Navigation,  Communication,  Call  for  Fires 

2.25 

52 

.79 

3.20 

22 

.89 

2.53 

74 

.92 

Weapons 

2.36 

53 

.86 

2.80 

22 

1.06 

2.49 

75 

.94 

Supervise,  Lead,  Train 

2.51 

52 

.90 

3.15 

22 

1.16 

2.70 

'74 

1.02 

To  facilitate  comparisons  between  the  supervisor  and  incumbent  ratings,  we  calculated 
the  mean  of  the  mean  task  ratings  within  each  category  (see  Tables  4  and  5).  With  the  exception 
of  the  NBC  Protection  and  Decontamination  ratings,  the  mean  supervisor  ratings  (both  time 
spent  and  test  emphasis)  were  higher  than  the  mean  incumbent  ratings.  The  differences  were 
particularly  pronounced  in  the  test  emphasis  ratings  for  four  task  categories — (a)  Vehicle  and 
Convoy  Operations;  (b)  Navigation,  Communication,  Call  for  Fires;  (c)  Weapons;  and  (d) 
Supervise,  Lead,  Train.  Given  the  small  number  of  supervisor  raters,  however,  and  the 
homogeneity  of  that  small  sample,  we  are  unable  to  conclude  that  these  patterns  of  differences 
would  hold  true  with  a  larger-scale  survey. 
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Table  5.  Category  Mean  Test  Emphasis  Ratings  by  Group 


Incumbent 

Supervisor 

Total 

M 

n 

SD 

M 

n 

SD 

M 

N 

SD 

First  Aid 

2.87 

53 

.80 

3.28 

22 

.74 

2.99 

75 

.80 

NBC  Protection  and  Decontamination 

2.75 

53 

.83 

2.44 

21 

1.04 

2.66 

74 

.90 

Vehicle  and  Convoy  Operations 

2.97 

53 

.82 

4.05 

22 

.73 

3.29 

75 

.93 

Threat  ID  and  Reaction 

3.03 

53 

1.09 

3.67 

22 

.59 

3.22 

75 

1.01 

Force  Protection 

2.84 

53 

1.01 

3.49 

22 

.73 

3.03 

75 

.98 

Urban  Operations,  Camouflage 

2.86 

53 

.92 

3.32 

22 

.59 

2.99 

75 

.86 

Navigation,  Communication,  Call  for  Fires 

2.78 

53 

1.08 

3.70 

22 

.67 

3.05 

75 

1.06 

Weapons 

2.74 

53 

.98 

3.62 

22 

.72 

3.00 

75 

.99 

Supervise,  Lead,  Train 

2.71 

52 

.98 

3.86 

22 

.89 

3.05 

74 

1.08 

We  looked  at  the  extent  to  which  the  time  spent  and  test  emphasis  ratings  corresponded 
with  each  other;  that  is,  were  tasks  that  E4  Soldiers  spend  most  time  on  those  judged  to  warrant 
more  coverage  on  an  E4  level  test?  The  correlation  between  time  spent  and  test  emphasis  ratings 
was  .41  for  supervisors  ( n  =  22,  p  =  .06)  and  .43  for  incumbents  ( n  =  53,  p  <  .001).  This  suggests 
that  the  two  judgments  were  only  moderately  related  to  one  another  for  both  types  of  respondents. 

We  also  looked  at  the  correlations  between  the  supervisor  and  incumbent  ratings.  The 
correlation  between  supervisor  and  incumbent  time  spent  ratings  was  .65  ( n  =  99,  p  <  .001)  and 
the  correlation  between  supervisor  and  incumbent  test  emphasis  ratings  was  .39  ( n  =  99 ,p< 
.001).  This  confirms  our  prior  observations  regarding  Tables  G.l  and  G.2 — that  there  was 
considerable  disagreement  between  the  two  groups  of  raters  regarding  what  should  be  covered 
on  a  test  and  less  disagreement  about  time  spent  on  various  tasks. 

Knowledge  Statement  Ratings 

Only  the  supervisors  rated  the  knowledge  statements,  and  they  used  just  a  single  test 
emphasis  scale.  The  mean  ratings  are  shown  in  Appendix  G  (Table  G.3).  These  ratings  are  more 
tightly  clustered  than  the  task  test  emphasis  ratings  provided  by  the  supervisors,  ranging  from 
3.05  (Requirements  of  Environmental  Regulations)  to  4.48  (Leadership  Duties).  Thus,  using  a 
rating  scale  with  relative  anchors  did  not  appear  to  help  spread  the  ratings.  This,  in  turn,  makes 
the  resulting  data  somewhat  less  useful  for  prioritizing  knowledge  areas  for  measurement. 

Reactions  to  Survey 

Survey  respondents  were  asked  to  answer  four  questions  about  the  survey,  and  their 
responses  are  summarized  in  Table  6.  As  we  might  expect,  the  incumbents  found  it  a  bit  harder 
than  supervisors  to  rate  the  tasks  (the  only  part  of  the  survey  the  incumbents  completed,  other 
than  the  background  and  survey  feedback  questions).  But  neither  group  indicated  the  tasks  were 
very  difficult  to  rate.  For  the  supervisors,  the  hardest  part  of  the  survey  was  expected  to  be 
weighting  the  five  test  categories  and  this  proved  to  be  true  given  this  received  the  highest 
difficulty  rating  (2.68  on  a  5-point  scale). 


22 


Table  6.  Respondent  Ratings  of  Difficulty  Completing  Survey 


Supervisors 

Incumbents 

M 

SD 

M 

SD 

Rating  tasks  on  time  spent 

2.00 

1.07 

2.41 

0.95 

Rating  tasks  on  test  emphasis 

2.22 

1.11 

2.41 

0.91 

Rating  knowledges  on  test  emphasis 

2.04 

1.21 

N/A 

N/A 

Weighting  categories 

2.68 

1.36 

N/A 

N/A 

Note.  1-5  scale,  where  1  =  very  easy  and  5  =  very  difficult.  Supervisor  n  =  22,  Incumbent  n  =  75. 

Survey  respondents  were  also  asked  for  open-ended  comments.  Based  on  many  of  the 
comments,  however,  it  is  evident  that  the  incumbents  (many  of  whom  took  the  survey  as  part  of 
the  PerformM21  pilot  test  activities)  confused  the  test  emphasis  survey  with  the  prototype  tests 
that  were  administered  in  the  same  session.  The  one  clear  message  was  that  the  incumbents 
thought  the  survey  was  too  long.  One  of  the  two  supervisor  comments  suggested  the  respondent 
was  trying  to  fit  the  rating  task  into  his  understanding  of  how  training  is  organized.  It  seems 
reasonable  to  assume  that  the  context  of  the  judgments  would  be  more  clearly  understood  were 
there  actually  an  operational  testing  program  for  which  the  input  would  be  directly  linked  in  the 
minds  of  survey  respondents. 


Blueprint  Development 
Initial  Category  Weights 

The  basic  process  for  transforming  test  emphasis  ratings  on  individual  task  and 
knowledge  statements  into  category  weights  for  a  test  blueprint  is  fairly  straightforward.  As 
shown  in  Table  7,  for  each  of  the  13  task  and  knowledge  categories,  we  began  by  summing  the 
mean  supervisor  ratings  for  each  task  and  knowledge  statement  in  the  category  (see  first 
column).  We  did  not  use  the  incumbent  ratings  because  our  original  intention  was  to  use  those 
only  for  comparison  to  the  supervisor  ratings,  not  for  blueprint  development.  Summing  the  mean 
ratings  has  the  effect  of  giving  greater  emphasis  to  categories  containing  more  tasks  and 
knowledges.  The  next  step  was  to  sum  those  figures  (see  bottom  of  the  first  column)  and  use  that 
as  a  basis  for  calculating  the  percent  of  the  total  accounted  for  by  each  category  (shown  in  the 
third  column).  For  example,  the  sum  of  the  mean  test  emphasis  ratings  of  the  tasks  in  the  First 
Aid  category  was  54.18,  which  is  13.85%  of  391.17.  The  fourth  column  in  Table  7  simply 
rounds  these  figures  to  the  closest  integer  (e.g.,  13.85  rounds  to  14).  The  test  weights  in  this 
column  total  100%,  so  the  actual  number  of  points  on  a  test  for  each  category  based  on  these 
weights  would  depend  on  the  total  number  of  points  on  the  test  (e.g.,  14  First  Aid  points  on  a 
100-item  test  or  28  points  on  a  200-point  test). 

Meta-Category  Weights 

Table  7  demonstrates  development  of  test  blueprint  weights  for  the  13  common  core 
categories  on  the  survey,  without  regard  for  the  five  meta-categories.  Assuming  we  want  a 
blueprint  that  makes  use  of  the  meta-categories,  there  are  two  ways  of  constructing  meta¬ 
category  weights  based  on  the  survey  data  we  collected.  First,  we  can  simply  total  the  weights 
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we  showed  in  Table  7  for  the  categories  comprising  each  meta-category.  We  refer  to  these  as 
“constructed”  weights  in  Table  8.  Second,  we  can  compute  the  mean  number  of  points  the 
supervisors  assigned  to  each  of  the  five  categories  in  Section  4  of  the  survey.  As  with  the 
individual  task  and  knowledge  ratings,  we  transformed  these  points  so  that  they  totaled  100%. 
These  are  referred  to  as  “holistic”  weights  in  Table  8.  For  comparison,  the  last  column  in  Table  8 
shows  the  results  of  a  similar  holistic  judgment  exercise  performed  by  the  PerformM21  ATP  AT 
members  prior  to  administration  of  the  survey. 


Table  7,  Category  Test  Emphasis  Weights,  Disregarding  Meta-Categories 


Category 

Sum  of  Mean 
Ratings 

Standard 

Deviation 

Percent  of 
Total 

Weight  (Rounded 
Percent  of  Total) 

1 .  First  Aid 

54.18 

.21 

13.85 

14 

2.  Weapons 

42.96 

.32 

10.98 

11 

3.  Force  Protection 

40.13 

.13 

10.26 

10 

4.  Urban  Operations 

36.36 

.17 

9.29 

9 

5.  Vehicle  &  Convoy  Ops 

29.83 

.14 

7.63 

8 

6.  Threat  ID 

29.52 

.23 

7.55 

8 

7.  Supervise 

25.19 

.13 

6.44 

6 

8.  Navigation 

24.83 

.18 

6.35 

6 

9.  NBC 

21.93 

.05 

5.61 

6 

10.  Leadership 

28.95 

.18 

7.40 

7 

1 1 .  Rules  &  Regulations 

21.76 

.39 

5.56 

6 

12.  Army  Policies 

20.96 

.35 

5.36 

5 

13.  Army  History,  Values 

14.57 

.14 

3.72 

4 

Total 

391.17 

100% 

100% 

Table  8.  Comparison  of  Meta-Category  Test  Emphasis  Weights  Derived  Alternative  Ways 


Meta-Category 

Constructed  Weight 
(Survey) 

Holistic  Judgment 
(Survey) 

Holistic  Judgment 
(ATPAT) 

Self-Protection  and  Defense 

28 

20 

23 

Weapons,  Comm,  Navigation 

17 

24 

23 

Tactical  Ops 

27 

21 

24 

Supervision  and  Training 

13 

23 

16 

Army  Roles  and  History 

15 

12 

14 

Total 

100% 

100% 

100% 

Army  Roles  and  History  received  the  lowest  blueprint  weight  regardless  of  how  it  was 
derived.  The  only  appreciable  difference  between  the  two  sets  of  holistic  weights  is  that  the 
survey  respondents  weighted  Supervision  and  Training  quite  a  bit  more  than  the  ATP  AT. 
Comparing  the  constructed  and  holistic  survey  weights  (which  were  based  on  input  from  the 
same  NCOs),  we  can  see  that  there  is  more  variation  across  categories  for  the  constructed 
weights.  This  may  be  an  artifact  of  the  holistic  judgment  process.  Think  about  dividing  100 
points  across  five  categories.  The  easiest  way  to  approach  this  is  to  start  with  equal  weights  (i.e., 
20  points  for  each  category)  and  adjusting  upward  and  downward  from  there.  This  might  result 
in  a  bias  toward  similar  weights  across  categories. 
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This  is  not  to  say  that  the  holistic  weights  are  less  valid  than  the  constructed  weights.  The 
constructed  weights  depend  in  significant  part  on  the  number  of  task  and  knowledge  statements 
within  each  category.  Although  we  strived  to  create  statements  that  would  be  equivalent  in  form,  it 
is  likely  that  some  statements  are  more  or  less  broad  than  others  (e.g..  First  Aid  tasks  appear  to  be 
more  finely  parsed  than  Supervision  tasks).  Examination  of  Table  8  provides  some  support  for  this 
hypothesis,  as  Supervision  and  Training  has  a  constructed  weight  of  13,  but  a  holistic  weight  of  23. 

A  Prototype  Blueprint 

Table  9  attempts  to  bring  the  categories  and  meta-categories  together  into  a  prototype 
blueprint.  That  is,  this  blueprint  will  yield  a  test  on  which  Soldiers  would  get  a  total  score  and 
five  subtest  scores.  We  chose  to  use  the  survey-based  holistic  meta-category  weights  from  Table 
8  as  our  subtest  weights.  We  then  determined  what  percentage  of  each  subtest  would  cover  each 
content  category  within  it  by  calculating  within-subtest  (meta-category)  weights. 

Even  with  large  samples  of  respondents,  there  will  be  some  sampling  error  that  will  lead 
to  small  non-substantive  differences  in  the  weights  derived  from  one  survey  to  the  next.  Earlier 
we  discussed  the  advantages  of  having  test  blueprints  that  are  relatively  stable  over  time,  so  such 
minor  changes  to  resulting  blueprint  weights  would  be  undesirable.  There  is  also  intuitive  appeal 
to  working  with  rounder  numbers.  Thus,  Table  9  shows  a  “smoothed”  blueprint  that  rounds  the 
subtest  weights  to  multiples  of  5. 


Table  9.  Prototype  Test  Blueprint 


Simple  Weights 

Smoothed  Weights 

Subtest 

Content  Category 

Weight  Within 
Subtest 

Meta-Category 

Weight 

Weight  Within 
Subtest 

Meta-Category 

Weight 

Self-Protection  and  Defense 

20 

20 

First  Aid 

51 

50 

NBC 

21 

20 

Threat  ID 

Weapons,  Comm,  Navigation 

28 

24 

30 

25 

Navigation 

63 

65 

Weapons 

Tactical  Ops 

37 

21 

35 

20 

Vehicle  &  Convoy  Ops 

28 

30 

Force  Protection 

38 

35 

Urban  Ops 

Supervision  and  Training 

34 

23 

35 

25 

Supervise 

46 

45 

Leadership 

Army  Roles  and  History 

54 

12 

55 

10 

Rules  &  Regulations 

38 

40 

Army  Policies 

37 

35 

Army  History,  Values 

25 

25 

Total  Test 

100 

100 

25 


We  skipped  one  step  in  construction  of  this  blueprint,  which  is  development  of 
definitions  for  each  of  the  test  content  categories.  Such  definitions  would  be  based  on  a  synthesis 
of  the  constituent  tasks  and  knowledges  into  an  easily  digested  yet  informative  summary  of  the 
content  covered  in  the  category.  An  example  of  such  a  format  for  the  First  Aid  category  is 
provided  below: 

First  Aid  refers  to  providing  emergency ,  life  saving  care  or 
treatment  to  Soldiers  who  are  injured  in  training  or  combat.  It 
encompasses  tasks  that  range  from  evaluating  injury  to  treating 
typical  combat  injuries  in  order  to  prevent  worsening  of  the 
situation  or  death,  including  transporting  and  evacuating 
personnel  It  also  includes  preventive  care  and  countermeasures. 

If  this  were  a  blueprint  destined  for  operational  use  (which  we  do  not  recommend  given 
the  small  number  of  NCOs  on  which  it  is  based),  the  next  step  would  be  to  review  the  draft 
blueprint  with  a  select  senior  group  of  NCOs  (a  test  council).  The  final  step  would  be  policy 
approval.  Some  adjustments  might  be  made  based  on  a  variety  of  considerations  at  these  stages, 
although  deviations  from  field  input  should  be  made  only  with  deliberate  and  careful 
consideration. 


Summary  and  Discussion 
Survey  and  Blueprint  Strategy 

Given  the  limited  number  of  individuals  who  responded  to  the  survey,  the  primary 
products  of  our  work  here  are  the  prototype  survey  itself,  and  the  elaboration  of  a  process  for 
using  survey  results  to  draft  a  test  blueprint.  Although  they  could  use  refinement  through  further 
SME  input,  the  task  and  knowledge  statements  and  associated  categories  and  meta-categories 
bring  the  Army  considerably  closer  to  the  foundation  of  a  common  core  test  blueprint  that 
reflects  deployment-related  lessons  learned  and  that  would  be  useful  for  an  operational  test 
program. 

It  is  quite  possible  that  the  survey  burden  could  be  reduced  by  collecting  only  test 
emphasis  ratings  from  supervisors  and  not  collecting  data  from  incumbents  (at  least  at  paygrade 
E4)  at  all.  We  are  reluctant  to  recommend  either  of  these  steps,  however,  until  enough  data  can 
be  collected  to  better  evaluate  the  implications  of  losing  this  additional  information. 

We  should  note  that  the  survey  we  developed  is  designed  to  support  development  of  a 
multiple-choice  test  blueprint,  again  with  a  liberal  definition  of  “multiple-choice”  meaning  any 
of  a  variety  of  selected  response  test  items.  The  survey  is  not  designed  to  help  determine  what 
test  methods  (e.g.,  multiple-choice,  hands-on,  situational  judgment,  computer-based  simulations) 
would  be  most  appropriate  to  develop,  nor  to  provide  the  additional  information  (e.g.,  critical 
incidents,  contextual  descriptors)  that  would  be  needed  to  design  these  alternative  types  of  tests. 
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Survey  Administration 


Our  survey  response  rate  was  very  poor,  even  given  that  response  rates  for  all  types  of 
surveys  have  been  dropping  over  the  last  decade.  The  Army’s  usual  approach  to  an  occupational 
analysis  survey,  which  we  were  not  able  to  implement  for  this  research,  involves  emailing  a 
survey  link  to  the  desired  population  (e.g.,  all  E4  Soldiers,  all  3 IBs)  using  Army  Knowledge 
Online  (AKO)  email  accounts.  Assuming  that  will  continue  to  be  the  Army’s  approach  for  their 
large-scale  training-oriented  occupational  surveys,  we  are  inclined  to  recommend  identification 
of  a  smaller  targeted  sample  of  respondents  for  test  emphasis  surveys.  Potential  respondents 
would  be  identified  to  help  ensure  desired  representativeness  (e.g.,  of  paygrades  and  MOS)  and 
experience  (e.g.,  with  deployments).  Then,  these  individuals  would  need  to  be  told  that  they  were 
specially  selected  to  participate  in  the  process.  This  approach  would  alleviate  the  security  and 
survey  length  issues  discussed  previously.  Another  goal  here  would  be  to  increase  individual 
motivation  to  participate.  Moreover,  if  a  test  emphasis  survey  is  linked  to  a  real  test  program, 
particularly  one  that  is  used  to  support  promotion  decisions,  it  is  likely  that  incumbents  and  their 
NCOs  will  be  more  interested  in  participating  in  the  survey  because  they  will  want  to  impact  the 
program. 
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CHAPTER  4:  ITEM  DEVELOPMENT 


Introduction 

Job  knowledge  test  item  development  typically  follows  a  standard  procedure:  (a)  conduct 
job  analysis,  (b)  develop  test  blueprint,  (c)  develop  items,  (d)  pilot  test  items,  and  (e)  develop  final 
test  form.  Chapter  3  discusses  the  first  two  steps,  and  this  chapter  is  concerned  with  the  third  step. 
We  did  not  attempt  to  pilot  test  these  items  or  create  new  test  forms  as  part  of  this  project. 

We  began  item  development  prior  to  completion  of  the  job  analysis  survey  because  of 
time  constraints.  There  were  three  sources  for  items:  (a)  Army-wide  items  developed  in  Phase  II 
of  the  PerformM21  project  (Knapp  &  Campbell,  2006);  (b)  Project  A  items  (J.C.  Campbell,  & 
Knapp,  2001);  and  (c)  items  developed  specifically  for  this  research.  Project  A  was  a  research 
effort  wherein  tests  were  developed  for  20  MOS.  Some  of  the  items  from  these  previously 
developed  tests  were  used,  updated,  or  otherwise  modified  for  incorporation  into  this  research 
(J.C.  Campbell  &  Knapp).  Because  the  job  analysis  survey  was  not  completed  when  item 
development  was  initiated,  we  did  not  have  a  test  blueprint  to  follow.  Instead,  we  used  the  job 
analysis  survey  task  and  knowledge  list  as  our  guide. 

Item  Development 

Our  first  step  was  to  sort  the  Army- wide  items  from  Phase  II  of  the  PerformM21  project 
into  the  survey  categories.  Phase  II  resulted  in  the  development  of  282  items,  of  which  70  were 
dropped  due  to  poor  item  statistics  during  Phase  III  pilot  testing.  We  were  able  to  sort  all  but  one 
of  the  remaining  items  into  the  tasks  and  knowledges.  We  then  adapted  20  Project  A  items, 
which  we  were  able  to  improve  by  upgrading  the  graphics  (e.g.,  changing  black  and  white 
graphics  to  color). 

For  new  item  development,  we  targeted  areas  we  believed  would  be  rated  as  most 
important  in  the  survey,  and  for  which  we  did  not  already  have  items.  These  areas  are  shown  in 
Table  10.  Comparison  with  Table  9  shows  that  we  made  good  choices. 

Table  10.  Targeted  Task/Knowledge  Areas  for  New  Item  Development 

Task/Knowledge  Area  Example  Task  or  Knowledge 

Vehicle  and  Convoy  Tactical  Operations  Occupy  floating  rally  point 

Threat  Identification  and  Reaction  -  Mines,  Snipers,  Ambush  React  to  ambush  -  dismounted 

Force  Protection,  Crowd  Control,  Security  and  Intelligence,  Detainee  Operations  Conduct  an  urban  patrol 
Urban  Operations,  Camouflage,  Fighting  Positions  Conduct  building  assaults 

Rules,  Regulations,  Laws  of  Conduct  Geneva  Convention 

Army  Policies  and  Practices  Equal  Opportunity  Policy 


As  with  the  other  PerformM21  efforts,  we  focused  on  developing  non-traditional  items 
(e.g.,  matching,  drag  and  drop,  and  matrix).  Item  development  is  an  iterative  process,  with  each 
item  undergoing  multiple  reviews  and  edits.  Item  development  may  proceed  with  items  being 
developed  by  SMEs  trained  by  test  developers,  or  by  item  developers  using  appropriate  reference 
material.  For  this  research,  56  items  were  developed  by  project  staff  using  various  technical 
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manuals  (TMs),  field  manuals  (FMs),  and  other  training  material,  often  material  developed 
specifically  for  OIF/OEF  operations.  Recall  that  some  of  the  source  material  had  an  FOUO 
designation;  thus,  items  developed  from  this  material  also  carried  an  FOUO  designation.  This 
means  that  extra  care  is  necessary  when  printing,  circulating,  or  publishing  the  items  (e.g., 
publishing  items  in  assessments  that  are  then  hosted  on  the  Internet).  In  all,  there  were  37  FOUO 
items. 


Item  Review 

As  mentioned,  items  are  often  reviewed  and  edited  multiple  times  to  ensure  they  are  clearly 
written  and  appropriate  for  the  testing  purpose.  The  items  adapted  from  Project  A  and  the  newly 
developed  items  were  reviewed  by  both  project  staff  and  Army  SMEs.  We  had  a  group  of  three 
NCOs  (one  at  the  E6  level  and  two  at  the  E7  level)  serve  as  our  SMEs.  We  recruited  them  with  the 
help  of  the  ATP  AT.  Each  SME  was  sent  a  packet  containing  the  76  items  (newly  developed  plus 
Project  A-adapted  items),  a  list  of  the  survey  categories,  and  instructions  detailing  how  to  review 
the  items.  The  following  factors  were  considered  during  item  review: 

•  Is  the  wording  of  the  stem  and  response  options  appropriate  for  our  target  test-taking 
population? 

•  Is  the  keyed  option  correct  and  are  the  unkeyed  options  incorrect? 

•  Is  the  item  content  still  applicable  or  current? 

•  Does  the  item  belong  in  the  blueprint  category  in  which  it  is  placed? 

•  Is  the  item  too  trivial? 

Based  on  the  SMEs  suggestions  and  input,  the  items  were  edited.  We  suggest  that  these 
newly-developed  items  be  reviewed  by  at  least  one  other  group  of  three  to  six  SMEs  prior  to 
being  pilot  tested.  Table  11  shows  the  number  of  items  in  each  blueprint  category. 

Most  item  development  efforts  will  also  include  content  validity  ratings,  which  assess  the 
items’  relevance  and  criticality  to  the  job.  For  an  operational  assessment,  it  would  be  best  to 
collect  these  ratings  from  three  to  six  judges  who  were  not  involved  in  the  item  development 
process.  The  judges  should  use  some  variation  of  the  following  scales  to  render  their  judgments: 


•  How  important  is  the  knowledge  or  skill  required  to  answer  this  item  for  acceptable 
performance?  Ratings  are  made  on  a  4-point  scale  ranging  from  “not  important”  to 
“very  important.” 

•  Lack  of  the  knowledge  or  skill  required  to  answer  this  item  could  result  in 
performance  errors  that  might  cause:  Options  range  from  “no  negative  consequences” 
to  “seriously  damaging  consequences.” 

We  did  not  collect  content  validity  ratings  for  any  items  included  in  this  project. 


Table  11.  Distribution  of  Items  Across  Prototype  Blueprint 


Content  Category 

Meta-Category 

Weight 

Number  of 
items3 

Self-Protection  and  Defense 

20 

First  Aid 

24 

NBC 

7 

Threat  ID 

24 

Weapons,  Comm,  Navigation 

25 

Navigation 

18 

Weapons 

64 

Tactical  Ops 

20 

Vehicle  &  Convoy  Ops 

23 

Force  Protection 

20 

Urban  Ops 

11 

Supervision  and  Training 

25 

Supervise 

21 

Leadership 

24 

Army  Roles  and  Flistory 

10 

Rules  &  Regulations 

10 

Army  Policies 

9 

Army  History,  Values 

27 

Total  Test 

100 

282 

a  Items  in  this  column  include  Army- wide  items  pilot  tested  in  Phase  II, 


adapted  Project  A  items,  and  newly  developed  items. 


Summary 

In  line  with  the  discussion  in  Chapter  2,  we  anticipated  that  some  of  these  task  and 
knowledge  areas  might  be  more  difficult  to  find  suitable  reference  material  for  than  others.  Not 
surprisingly,  it  was  fairly  easy  to  find  reference  material  for  Rules,  Regulations,  Laws  of  Conduct 
and  Army  Policies  and  Practices.  These  are  not  areas  subject  to  change  based  on  theater  of 
operations.  Chapter  2  noted  that  much  of  the  material  was  theater-  or  area-specific.  For  example 
improvised  explosive  devices  (IEDs)  are  more  of  a  problem  in  Fallujah  and  along  the  Syrian 
border  than  in  the  south  of  Iraq.  Soldiers  deployed  to  those  areas  are  likely  to  have  different 
viewpoints  on  what  kind  of  a  threat  IEDs  are  and  how  to  prevent  or  locate  them.  Indeed,  the 
information  can  even  be  time-specific.  For  example,  suicide  bombings  were  more  of  a  problem  in 
the  summer  than  fall  of  2005.  The  SME  reviews  support  these  comments  as  well.  The  SMEs  had 
been  deployed,  but  to  different  places  and  at  different  times.  In  reviewing  the  items,  some  agreed 
that  tasks  were  to  be  performed  in  a  certain  manner  whereas  others  disagreed.  For  instance,  one  of 
the  CALL  website  documents  specified  the  order  in  which  Soldiers  are  to  exit  and  re-enter  a 
vehicle  that  has  come  under  attack.  Part  of  the  SME  group  agreed  and  part  indicated  order  did  not 
matter. 


The  theater-specificity  of  some  tasks  and  knowledge  areas  has  consequences  for 
developing  and  maintaining  an  Army-wide  assessment  of  tasks  and  knowledges  that  reflect  the 
current  operating  environment.  There  is  likely  to  be  some  need  to  update  item  content  and/or 
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scoring  if,  for  example,  the  order  of  steps  changes  from  one  theater  to  another.  An  example  from 
Chapter  2  is  the  steps  for  conducting  a  trauma  assessment.  The  “standard”  way  to  conduct 
trauma  assessment  is  airway,  breathing,  circulation  (ABC).  However,  because  of  the  types  of 
injuries  they  are  seeing  in  Iraq,  the  operational  sequence  is  circulation,  airway,  breathing  (CAB). 
An  item  keyed  according  to  the  standard  way  to  conduct  trauma  assessment  would  have  to  be 
either  re-keyed  to  reflect  the  operational  reality  (for  some  Soldiers)  or  the  item  would  have  to  be 
qualified  in  some  manner  such  as,  “Based  on  how  you  were  taught  in  AIT  (or  Iraq),  the  correct 
sequence  for  conducting...” 

It  is  also  possible  that  an  assessment  blueprint  for  Soldiers  in,  or  soon  to  be  in,  Iraq  would 
be  different  from  one  for  Soldiers  in  Afghanistan  or  Korea.  These  examples  illustrate  why 
conducting  a  comprehensive  survey  such  as  the  prototype  developed  for  this  research  is  so 
important.  Of  course,  for  an  operational  survey,  a  much  larger  sample  size  to  ensure  more  voices 
were  heard  would  be  of  paramount  importance. 
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CHAPTER  5:  SUMMARY  OF  RESULTS 


Recall  that  the  PerformM21  effort  was  a  3-year  feasibility  effort  to  identify  viable 
approaches  for  the  development  of  a  useful  yet  affordable  operational  Soldier  performance 
assessment  system  (Knapp  &  Campbell,  2006).  The  research  goal  was  to  further  the  work  started 
under  the  PerformM21  project  to  include  understanding  of  how  recent  deployment  activities  can 
and  should  influence  Army  assessment  program  content.  This  was  accomplished  by  developing  a 
prototype  survey  for  determining  suitable  content  for  an  operational  common  core  examination 
and  developing  new  test  items  covering  areas  especially  pertinent  to  recent  deployment  activities 
in  the  Middle  East. 

Our  work  began  with  development  of  task  and  knowledge  lists  intended  to 
comprehensively  capture  potential  test  content  for  all  E4  Soldiers,  including  those  serving  in  a 
deployed  capacity.  Challenges  included  (a)  obtaining  material  representing  recent  lessons 
learned  from  sources  that  have  not  been  well-processed  or  well-documented,  (b)  identifying 
material  that  is  reasonably  applicable  across  time  and  settings,  and  (c)  identifying  doctrinal 
references  for  recent  lessons  learned.  Without  established  doctrine,  it  would  not  be  reasonable  to 
hold  Soldiers  accountable  for  the  material;  however,  the  Army  is  increasingly  becoming  an 
organization  in  which  the  field,  rather  than  the  schoolhouses,  is  establishing  doctrine.  Another 
challenge  was  development  of  test  content  categories  that  adequately  serve  testing  needs  (e.g., 
for  suitably  long  subtests)  while  avoiding  combining  unrelated  content  areas. 

The  prototype  test  emphasis  survey  and  associated  test  blueprint  development  procedures 
demonstrate  how  field  input  can  be  efficiently  and  directly  translated  into  the  design  for  a 
content  valid  test  that  reflects  the  experience  of  a  broad  range  of  Soldiers,  including  those  who 
have  recent  deployment  experience  in  a  variety  of  regions.  Although  we  were  unable  to  collect 
data  from  many  respondents,  the  survey  and  associated  procedures  are  robust  and  can  be  adapted 
for  future  use  very  easily.  It  would  be  desirable  to  have  further  field  input  into  the  task  and 
knowledge  lists,  but  we  expect  the  changes  would  be  relatively  minor.  Indeed,  repeating 
administration  of  the  current  survey  to  gather  input  from  a  larger  sample  of  respondents  would 
provide  the  Army  with  a  reasonable  working  operational  test  blueprint. 

Project  staff  developed  new  test  items  to  tap  some  of  the  content  areas  that  seem 
particularly  relevant  to  recent  deployment  operations.  Combining  these  common  core  items  with 
those  developed  in  PerformM21  yields  a  bank  of  282  test  items  that  the  Army  now  has  to  support 
future  test  activities.  This  is  a  sizable  starting  test  bank,  although  it  should  be  pointed  out  that 
content  validity  ratings  should  be  collected  on  all  of  these  items,  and  some  of  them  still  need  to 
be  pilot  tested. 

In  summary,  despite  the  challenges  associated  with  incorporating  recent  lessons  learned 
in  deployment  operations  into  an  Army  test  program,  we  believe  that  it  is  not  only  possible  to  do 
so,  but  vital  that  this  be  done.  Both  the  credibility  and  effectiveness  of  testing  are  dependent  on 
the  job-relevance  of  test  content,  which  can  be  ensured  through  frequent  and  carefully  collected 
input  from  a  variety  of  available  resources  (e.g.,  the  CALL  website)  and  input  from  the  field  (as 
illustrated  through  our  prototype  job  analysis  survey). 
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Appendix  A 

Army-Wide  Common  Core  Tasks  and  Knowledge  Categories 


Common  Tasks:  Skill  Level  1 

1.  First  Aid. 

2.  M16  rifle/M4  carbine/M9  pistoL 

3.  Nuclear,  Biological,  and  Chemical  (NBC). 

4.  Communicate:  Radio  and  telephone. 

5.  Combat  Techniques  (Survive). 

6.  Navigate,  mounted  and  dismounted. 

7.  Defense  Measures:  camouflage,  SALUTE  (size,  activity,  location,  unit,  time, 
equipment),  operational  security  (  OPSEC). 

8.  M60  machine  gun/  M249  squad  automatic  weapon  (SAW)/M240B  machine  gun. 

9.  Individual  Conduct  and  Laws  of  War. 

10.  Hand  grenades  and  land  mines. 

11.  Remains  reporting  and  handling. 

12.  Caliber  .50  M2  machine  gun. 

13.  M203  40mm  grenade  launcher. 

14.  MK19  40mm  grenade  launcher  machine  gun. 

15.  Ml 36  launcher/ AT4  light  anti-tank  weapon. 

Common  Tasks:  Skill  Level  2 

1.  Combat  techniques  (Survive). 

2.  First  Aid:  Medical  evacuation  (MEDEV AC), preventive  medicine. 

3.  Equipment  and  maintenance:  preventive  maintenance  checks  and  services  (PMCS), 
supply  discipline,  property  accountability. 

4.  Defense  measures:  Squad  defense. 

5.  Navigate:  Map  overlays. 

6.  Risk  Management:  Accident  prevention. 

7.  Nuclear,  Biological,  and  Chemical  (NBC). 

Leadership 

1.  Identify  the  leadership  duties,  responsibilities,  authority,  and  requirements  of  officers  and 
NCOs. 

2.  Know  the  policies  and  procedures  of  the  chain  of  command  and  of  the  NCO  support 
channel. 

3.  Identify  the  principles  of  BE,  KNOW,  DO. 

4.  Know  the  principles  of  discipline. 

5.  Identify  the  risk  management  process. 

6.  Identify  the  steps  of  assuming  a  leadership  position. 

7.  Know  the  NCO  Education  System  (NCOES). 

8.  Know  the  characteristics  of  effective  counseling. 

9.  Understand  the  principles  of  the  rules  of  engagement  (ROE)  and  use  of  force. 

10.  Know  the  steps  in  troop  leading  procedures  (TLP). 
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11.  Prepare  for  and  conduct  inspections  and  perform  on-the-spot  corrections. 

12.  Resolve  an  ethical  problem. 

13.  Know  the  problem  solving  steps  in  decision-making. 

14.  Understand  the  requirements  of  the  semi-centralized  (E5-E6)  promotion  system. 

15.  Know  the  principles  of  financial  management. 

16.  Identify  the  Army’s  homosexual  policy. 

Training 

1.  Train  subordinates  to  perform  an  individual  task. 

2.  Prepare  for  and  conduct  preparatory  marksmanship  training  (PMT). 

3.  Identify  the  roles  and  responsibilities  of  the  NCO  in  training. 

4.  Prepare  for  and  conduct  drill  and  ceremonies. 

5.  Conduct  performance  oriented  training. 

6.  Prepare  for  and  conduct  crew  drills. 

7.  Identify  the  roles  and  functions  of  initial  training,  entry  training  (IET)  institutional 
training  (IT),  self-development  (SD)  training,  professional  military  education  (PME),  and 
unit  training  UT). 

8.  Know  the  principles  and  applications  of  Army  training  time  management  (Green,  Amber, 
Red). 

9.  Identify  the  content,  uses,  sources  and  locations  of  training  publications,  Army  Test  and 
Evaluation  Program-Mission  Training  Plan  (ARTEP-MTP),  field  manuals  (EM), 
technical  manuals  (TM),  Soldier  training  publications  (STP),  training  support  packages 
(TSP),  tactics,  techniques  and  procedures  (TTP),  and  lessons  learned  (LL). 

10.  Know  the  principles  of  Battle  Focused  Training. 

1 1 .  Conduct  training  preparation. 

12.  Prepare  for  and  conduct  Sergeant’s  Time  and  opportunity  training. 

13.  Prepare  a  training  outline/lesson  plan. 

Army  History  and  Customs:  Army  Values 

1.  The  Army  in  Colonial  Times:  US  Militia,  the  Revolutionary  War,  the  War  of  1812,  the 
Mexican  War. 

2.  The  Army  from  the  Civil  War  to  WWI:  the  Civil  War,  the  frontier  settlements,  the 
Spanish-American  War. 

3.  The  Army  in  the  World  Wars  and  Containment:  WWI,  between  the  wars,  WWII,  the 
Cold  War,  Korea,  Viet  Nam. 

4.  The  Army  post-Viet  Nam  and  the  Volunteer  Army:  the  Volunteer  Army,  the  end  of  the 
Cold  War,  the  Gulf  War,  relief  in  Africa,  Haiti,  the  Balkans,  the  War  on  Terrorism,  the 
Second  Gulf  War. 

5.  The  Army  in  the  Contemporary  Operational  Environment:  the  increasing  range  of  threats, 
full  spectrum  operations,  Homeland  Security,  the  Army  Transformation. 

6.  The  Army  Customs  and  Courtesies:  the  hand  salute,  bugle  calls,  the  Army  Flag. 

7.  The  Seven  Army  Values. 
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Appendix  B 

Warrior  Tasks  and  Battle  Drills  -  Initial  Entry  Training  (IET) 


TASKS 

Shoot 

1 .  Qualify  with  assigned  weapon. 

2.  Correct  malfunctions  with  assigned  weapon. 

3.  *Engage  targets  with  M240B  machine  gun. 

4.  *  Engage  targets  with  M249  (SAW)  machine  gun 

5.  *Engage  targets  with  M2  .50  caliber  machine  gun. 

6.  *Engage  targets  with  MK19  machine  gun. 

7.  Correct  malfunctions  on  an  M2  .50  caliber  machine  gun. 

8.  Correct  malfunctions  on  an  M240B  machine  gun. 

9.  Correct  malfunctions  on  an  M49  (SAW)  machine  gun. 

10.  Correct  malfunctions  on  an  MK19  machine  gun. 

11.  Engage  targets  with  weapon  using  night  vision  sight  AN/PVS-4. 

12.  ^Engage  targets  with  weapon  using  night  vision  sight  AN/PAS- 13. 

13.  *Engage  targets  with  weapon  using  night  vision  sight  AN/TVS-5. 

14.  Engage  targets  using  aiming  light  AN/PEQ-2A. 

15.  Engage  targets  using  aiming  light  AN/PAQ-4. 

16.  Employ  mines  (manned)  and  hand  grenades. 

Communicate 

17.  Perform  voice  communications:  Situation  Report  (SITREP),  Spot  Report  (SPOTREP). 

18.  Perform  voice  communications:  Call  for  Fire. 

19.  *Perform  voice  communications:  Medical  Evacuation  (MEDEVAC). 

20.  Use  visual  signaling  techniques. 

Joint  Urban  Operations 

21.  Perform  movement  techniques  during  an  urban  operation. 

22.  Engage  targets  during  an  urban  operation. 

23.  Enter  a  building  during  an  urban  operation. 

Move 

24.  Determine  location  on  the  ground  using  terrain  association,  map,  and  global  positioning 
system  (GPS). 

25.  Navigate  from  one  point  to  another  dismounted. 

26.  Move  over,  through,  or  around  obstacles,  except  minefields. 

27.  Prepare  a  vehicle  for  a  convoy  operation. 

Fight 

28.  Move  under  direct  fire. 

29.  React  to  indirect  fire,  dismounted  and  mounted. 

30.  React  to  direct  fire,  dismounted  and  mounted. 

31.  React  to  unexploded  ordnance  (UXO)  hazard. 
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32.  React  to  man-to-man  contact  (combatives). 

33.  React  to  chemical  or  biological  attack  or  hazard. 

34.  Decontaminate  your  self  and  individual  equipment  using  chemical  decontaminating  kits. 

35.  Maintain  equipment. 

36.  Evaluate  a  casualty. 

37.  Perform  first  aid  for  open  wound  -  abdominal,  chest,  head. 

38.  Perform  first  aid  for  bleeding  of  an  extremity. 

39.  Select  temporary  fight  position. 

BATTLE  DRILLS 

1.  React  to  contact:  visual,  improvised  explosive  device  (IED),  direct  fire  (including  rocket 
propelled  grenades  [RPG]). 

2.  *React  to  ambush  -  blocked. 

3.  *React  to  ambush  -  unblocked. 

4.  React  to  indirect  fire. 

5.  React  to  chemical  attack. 

6.  Break  contact. 

7.  *Dismount  a  vehicle. 

8.  *Evacuate  injured  personnel  from  vehicle. 

9.  Secure  at  a  halt. 

*Task  or  battle  drill  currently  not  being  trained  to  standard  in  IET  -  familiarization  training 
only. 
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Appendix  C 

FY06  Common  Task  Test  -  Skill  Levels  One  and  Two 


Skill  Level  1 


Task  1:  (Commander  chooses  one  of  the  following  three  tasks): 

1.  *Maintain  an  M9  pistol. 

2.  ^Maintain  an  M4  or  M4G.  1  carbine. 

3.  *Maintain  an  Ml 6- series  rifle. 

Task  2:  (Commander  chooses  one  of  the  following  five  tasks): 

1.  *Maintain  an  M2  .50  caliber  machine  gun. 

2.  *Maintain  an  M240B  machine  gun. 

3.  *Maintain  an  MK19  machine  gun. 

4.  Maintain  an  M60  machine  gun. 

5.  *Maintain  an  M249  (SAW)  machine  gun. 

Task  3:  *React  to  a  possible  improvised  explosive  device  (IED). 

Task  4:  *Move  under  direct  fire. 

Task  5:  *Evaluate  a  casualty  (tactical  combat  casualty  care). 

Task  6:  ^Perform  first  aid  for  a  bleeding  or  severed  extremity. 

Tasks  7-12:  (Commander  chooses  six  of  the  following  twelve  tasks): 

1.  Detect  explosive  hazard  indicators  by  visual  means. 

2.  *  React  to  indirect  fire  while  dismounted. 

3.  ^Navigate  from  one  point  on  the  ground  to  another  point  while  dismounted. 

4.  *  React  to  direct  fire  while  mounted. 

5.  Perform  first  aid  to  prevent  or  control  shock. 

6.  Practice  individual  preventive  medicine  countermeasures. 

7.  Communicate  via  a  tactical  radio  in  a  secure  net. 

8.  Search  vehicles  in  a  tactical  environment. 

or 

Search  a  vehicle  for  explosives  or  prohibited  items. 

9.  Employ  progressive  levels  of  individual  force  when  confronting  civilians. 

10.  Control  entry  to  and  exit  from  a  restricted  area. 

or 

Control  access  to  a  military  installation. 

11.  *Report  intelligence  information. 

12.  Operate  a  vehicle  in  a  convoy. 
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Skill  Level  2 


Tasks  13-15:  (Commander  chooses  three  of  following  six  tasks): 

1.  Establish  an  observation  post. 

2.  *Request  medical  evacuation. 

3.  ^Evaluate  casualties. 

4.  ^Conduct  a  presence  patrol. 

5.  Conduct  combat  operations  according  to  the  Law  of  War. 

6.  Employ  accident  prevention  measures  and  the  risk  management  process. 

^Indicates  task  corresponds  to  a  task  in  the  Warrior  Task  and  Battle  Drill  list. 
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Appendix  D 

TRADOC-ARI  Common  Tasks  Survey  -  Skill  Levels  One  and  Two 


Skill  Level  1 


Task  Categories  and  Number  of  Tasks 

1.  Individual  Conduct:  19  tasks. 

2.  First  Aid:  23  tasks. 

3.  Nuclear,  Biological,  Chemical  (NBC):  25  tasks. 

4.  Unit  Operations:  62  tasks. 

5.  Land  Navigation:  8  tasks. 

6.  Communications:  10  tasks. 

7.  Security  and  Intelligence:  13  tasks. 

8.  Crowd  and  Riot  Control:  4  tasks. 

9.  Defensive  Measures:  24  tasks. 

10.  Leadership:  3  tasks. 

11.  M 16  Series  Rifle:  7  tasks.  ,  .• 

12.  M4  Series  Carbine:  8  tasks. 

13.  M9,  9mm  Piston:  4  tasks. 

14.  Ml 36  Anti  Tank  Launcher  -  AT-4:  3  tasks. 

15.  Hand  Grenades  and  Grenade  Launchers  (M203/MK19):  11  tasks. 

16.  Shotgun:  3  tasks. 

17.  M240B  Machine  Gun:  10  tasks. 

18.  M249  (SAW)  Machine  Gun:  8  tasks. 

19.  M60  Machine  Gun:  12  tasks. 

20.  M2,  .50  caliber  Machine  Gun:  1 1  tasks. 

21.  MK19  Machine  Gun:  8  tasks. 


Skill  Level  2 


Task  Categories  and  Number  of  Tasks 

1.  First  Aid:  6  tasks. 

2.  Nuclear,  Biological,  Chemical  (NBC):  25  tasks. 

3.  Unit  Operations:  38  tasks. 

4.  Land  Navigation:  3  tasks. 

5.  Communications:  6  tasks. 

6.  Defensive  Measures:  1 1  tasks. 

7.  Administrative:  12  tasks. 

8.  Training:  4  tasks. 

9.  Maintenance  and  Supply:  6  tasks. 

10.  Risk  Management  and  Force  Protection:  8  tasks. 

11.  Military  Justice:  4  tasks. 

12.  Leadership:  19  tasks. 


D-l 


Appendix  E 

CALL-Based  Categories,  Knowledges,  Topics,  and  Tasks 


Category:  People,  Culture,  Sociology,  Area 

1.  Iraqi  History  -  People:  Turkomans,  Arabs,  Kurds. 

2.  History  -  Religion:  Shi’ a,  Sunni,  Islam,  Christian. 

3.  Iraqi  Calendar  and  Holidays. 

4.  Iraqi  Customs  and  Culture:  Women,  food,  family,  dress,  hospitality. 

5.  History  -  Politics:  Parties  and  groups. 

6.  Iraq  -  Key  political  figures. 

7.  Regime  loyalists. 

8.  Paramilitary  groups. 

9.  Fundamentalist  groups. 

10.  Afghani  -  Key  political  figures. 

11.  Afghani  -  Key  political  parties  and  groups. 

12.  Iraqi  Army. 

13.  Afghani  Army. 

14.  Coalition  Forces. 

15.  Non-Governmental  Organizations  (NGOs),  International  Organizations  (IOs),  Private 
Volunteer  Organizations  (PVOs). 

16.  Arabic  Language:  Key  words,  phrases,  greetings,  operational  terms,  numbers,  signs. 

17.  Iraq  -  key  provinces  and  cities;  facilities. 

18.  Afghanistan  -  key  regions,  cities,  facilities. 

19.  Iraq  -  terrain  and  weather. 

20.  Afghanistan  -  terrain  and  weather. 

Medical:  Preventive  Medicine  and  First  Aid 

1.  Handwashing,  cleanliness. 

2.  Eating  and  drinking. 

3.  Fitness. 

4.  Sleep. 

5.  Insect/vector  repellent  and  protection. 

6.  Diarrhea. 

7.  Upper  respiratory  illness  (URI). 

8.  Leishmaniasis. 

9.  Sand  fly  fever. 

10.  Malaria. 

11.  Dengue. 

12.  Leptospirosis. 

13.  Schistosomiasis. 

14.  Typhoid. 

15.  Paratyphoid  enteric  fever. 

16.  Rabies. 

17.  Carbon  monoxide  poisoning. 

18.  Heat  injuries. 
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19.  Cold  injuries. 

20.  Trench  foot. 

21.  Snake  and  insect  bites. 

22.  Acute  mountain  sickness  (AMS). 

23.  High  altitude  cerebral  edema  (HACE). 

24.  High  altitude  pulmonary  edema  (HAPE). 

25.  Diomax  prophylactic  treatment  administration  (mountain  operations). 

26.  Hearing  loss. 

27.  Eye  protection/injury. 

28.  Combat  stress. 

29.  Suicide  prevention. 

30.  Trauma  assessment  (circulation,  breathing,  air  way  -  CBA). 

31.  Self  (one- hand)  tourniquet  application. 

32.  Tourniquet  application  (precedes  pressure  dressing). 

33.  Use  of  Emergency  Trauma  Dressing. 

34.  Use  of  Kurlex. 

35.  Treatment  of  penetrating  neck  injuries. 

36.  Identification  of  tension  pneumothorax. 

37.  Use  of  the  folding  litter  -  half  fold  and  quad  fold. 

38.  Use  of  Quick  Clot. 

39.  Use  of  the  Combat  Pill  Pack  (Tylenol,  COX-2  inhibitor,  Gatifloxacin). 

40.  Casualty  evacuation  (CASEVAC)/medical  evacuation  (MEDEVAC);  use  of  9-line 
MEDEVAC. 

Vehicle  and  Convoy  Operations 

1 .  Prepare  vehicle  for  convoy  operations. 

2.  Conduct  convoy  pre-combat  checks. 

3.  Conduct  vehicle/convoy  troop  leading  procedures. 

4.  Perform  driver  side  contact  vehicle  battle  drill. 

5.  Perform  truck  commander  contact  vehicle  battle  drill. 

6.  Perform  exit/enter  vehicle  battle  drill. 

7.  Perform  exit/enter  cargo  area  vehicle  battle  drill. 

8.  Perform  react  to  contact/threat  (maintain  movement)  convoy  battle  drill. 

9.  Perform  forced  stop  convoy  battle  drill. 

10.  Conduct  CASEVAC/recovery  operations. 

11.  Perform  break  contact  convoy  battle  drill. 

12.  Occupy  a  floating  rally  point. 

13.  Prepare  improvised  explosive  device  (IED)/unexploded  ordnance  (UXO)  reports. 

14.  Use  vehicle/convoy  visual  (hand  and  arm)  signals. 

Urban  Combat  Operations 

1.  Conduct  lop  down  and  bottom  up  building  assaults. 

2.  Establish  urban  footholds  in  combat  operations. 

3.  Clear  a  structure  -  entry  and  breaching. 

4.  Take  actions  on  contact  -  urban  operations. 

5.  Employ  supporting  fires  -  urban  operations. 
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6.  Employ  demolitions  -  composition  four  (C4). 

7.  Construct  and  employ  Molotov  cocktails. 

8.  Construct  and  employ  propane  fuel-air  explosives. 

Combat  Operations  and  Procedures 

1 .  Operate  a  vehicle  check  point. 

2.  Conduct  entry  control  point  (ECP)  operations. 

3.  Perform  vehicle  search  procedures. 

4.  Conduct  IED  reaction  drills. 

5.  Perform  graduated  response  for  traffic  movement. 

6.  Identify  IED/VBIED  threats. 

7.  React  to  a  hostile  crowd. 

8.  Employ  reflexive  fires. 

9.  React  to  ambush. 

10.  React  to  sniper. 

11.  Operate  Handheld  Standoff  Mine  Detector  System  (HSTAMIDS). 

12.  Know  and  identify  anti-coalition  militia  (ACM)  tactics,  techniques,  and  procedures 
(TTP). 

13.  Interact  with  battlefield  media. 

14.  Know  and  apply  Law  of  War  and  Coalition  Forces  Land  Component  Command  (CFLCC) 
Rules  for  Use  of  Force. 


Appendix  F 

Lessons  Learned  List  of  Tasks  and  Knowledges 


Task  List 


First  Aid.  Preventive  Medicine,  and  Medical  Evacuation 

1.  Administer  first  aid  for  nerve  agent  casualty  (buddy  aid). 

2.  Evacuate  injured/wounded  from  a  vehicle. 

3.  Evaluate  a  casualty. 

4.  Initiate  an  intravenous  infusion. 

5.  Practice  preventive  medicine  -  hygiene,  sleep,  environment,  insects,  food  and  drink, 
weather. 

6.  Perform  fist  aid  for  an  open  wound  -  abdominal,  chest,  head. 

7.  Perform  first  aid  -  bleeding/traumatic  amputation/apply  combat  application  tourniquet 
(CAT). 

8.  Perform  first  aid  -  burns. 

9.  Perform  first  aid  -  cold  injuries. 

10.  Perform  first  aid  -  heat  injuries. 

11.  Perform  first  aid  -  shock. 

12.  Perform  first  aid  -  fracture. 

13.  Perform  first  aid  -  throat  obstruction. 

14.  Perform  mouth- to-mouth  resuscitation. 

15.  Put  on  a  field/pressure/emergency  trauma  dressing. 

16.  Request  a  medical  evacuation  (medical  evacuation/casualty  evacuation  - 
MEDEVAC/CASEVAC). 

17.  Transport  a  casualty. 

18.  Recover  remains. 

NBC  Protection  and  Decontamination 

1.  Cross  a  contaminated  area  -  chemical  or  radiological. 

2.  Decontaminate  equipment. 

3.  Decontaminate  self/skin. 

4.  Detect  chemical  agents. 

5.  Put  on  and  wear/operate  in  protective  mask  and  mission  oriented  protective  posture 
(MOPP)  gear  -  eat,  drink,  sleep,  waste. 

6.  Exchange  MOPP  gear. 

7.  Maintain  protective  mask/change  canister. 

8.  React  to  attack  or  hazard  -  chemical,  radiological,  biological. 

Vehicle  and  Convoy  Tactical  Operations 

1.  Perform  vehicle  preventive  maintenance  checks  and  services  (PMCS)  -  before,  during 
after  operations. 

2.  Prepare  for  convoy/movement  vehicular  operations. 

3.  Drive  vehicle,  with  or  with  out  trailer  -  day,  night,  black-out,  convoy,  off-road. 
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4.  Provide  convoy  in-transit  security. 

5.  Secure  vehicles/convoy  at  a  halt. 

6.  Perform  exit/enter  vehicle  battle  drill. 

7.  Perform  forced  stop  convoy  battle  drill. 

8.  Perform  break  contact  convoy  battle  drill. 

9.  Occupy  a  floating  rally  point. 

Threat  Identification  and  Reaction  -  Mines.  Sniper,  Ambush 

1.  Identify  threats  visually  and  by  other  indicators  -  mines,  improvised  explosive  device 
(IED),  vehicle  borne  improvised  explosive  device  (VBIED),  booby  traps. 

2.  Locate  mines  by  probing. 

3.  Mark/report  unexploded  ordnance  (UXO)  hazards. 

4.  Conduct  self-extraction  from  a  mined  area. 

5.  React  to  contact  (mounted  or  dismounted)  -  attack,  indirect  fire,  mines,  IED,  VBIED, 
rocket  propelled  grenade  (RPG). 

6.  Move  under  direct  fire. 

7.  React  to  sniper. 

8.  React  to  ambush  (mounted)  -  blocked  or  unblocked. 

9.  React  to  ambush  -  dismounted. 

Force  Protection,  Crowd  Control.  Security  and  Intelligence,  Detainee  Operations 

1.  Operate  a  vehicle/personnel  control  checkpoint  -  search  and  clear  vehicles/personnel. 

2.  Construct  non-explosive  approach  and  control  obstacles. 

3.  Conduct  entry  control  point  (ECP)  operations. 

4.  Conduct  intrusion  prevention  and  control  measures. 

5.  Conduct  an  urban  patrol  -  dismounted  or  mounted. 

6.  Visually  identify  vehicles  and  aircraft  -  friend  or  foe. 

7.  Establish  and  occupy  an  observations  post  (OP). 

8.  Conduct  surveillance/collect,  report  intelligence  information. 

9.  Conduct  crowd/riot  control  operations/formation/tactics. 

10.  React  to  man-to-man  contact  (combatives)/subdue  prisoner/detainee. 

1 1.  Identify/secure/search/segregate/safeguard/process  -  non-combatants/retained  personnel. 
(RP)/civilian  internees  (CI)/enemy  prisoners  of  war  (EPW). 

12.  Process  captured  or  suspect  material,  weapons,  equipment. 

13.  Recognize  and  process  captured  or  found  intelligence  items. 

Urban  Operations,  Camouflage,  Fighting  Positions 

1.  Perform  fire/maneuver  during  urban  tactical  operations/employ  supporting  fires. 

2.  Execute  building  breach  -  explosive,  mechanical,  shotgun. 

3.  Conduct  building  assaults  -  top  down  or  bottom  up. 

4.  Prepare  individual  and  crew-served  weapons  positions  -  urban  operations. 

5.  Search/secure  a  building/room. 

6.  Practice  noise/light/litter  discipline. 

7.  Select/prepare/hasty/temporary  fighting/weapons  positions  -  urban/non-urban. 

8.  Move  over,  through,  around  obstacles  (dismounted)  (except  minefields). 

9.  Clear  fields  of  fire. 
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10.  Camouflage  self. 

11.  Camouflage  equipment/fighting  positions. 

12.  Prepare  a  defensive  position  range  card. 


Navigation,  Communication.  Call  for  Fires 

1.  Determine  direction/location  on  the  ground  -  field  expedient,  map,  global  positioning 
system  (GPS),  terrain  association. 

2.  Navigate  from  one  point  to  another  on  the  ground  using  map/GPS/non-GPS  - 
mounted/dismounted. 

3.  Locate  a  target  by  grid  coordinates. 

4.  Call  for  and  adjust  indirect  fire. 

5.  Set  up,  operate,  and  communicate  over  tactical/commercial  radios. 

6.  Perform  voice  communications  -  send  situation  report  (SITREP)/spot  report  (SPOTREP). 

7.  Construct  field  expedient  antennae. 

8.  Use  visual  signaling  techniques/react  to  arm  and  hand  signals. 

Weapons,  Grenades,  Mines,  and  Demolitions 

1.  Employ  and  recover  antipersonnel  mines. 

2.  Employ  and  recover  antitank  mines. 

3.  Employ  demolitions  -  composition  four  (C4),  trinitrotoluene  (TNT),  blasting  caps, 
detonating  cord. 

4.  Maintain,  operate,  and  engage  targets  -  M16  series  rifle/M4  carbine. 

5.  Maintain,  operate,  and  engage  targets  -  M9,  9mm  pistol. 

6.  Maintain,  operate,  and  engage  targets  -  Ml 36  (AT4)  light  anti-tank  weapon. 

7.  Maintain,  operate,  and  engage  targets  -  M203  grenade  launcher. 

8.  Maintain,  operate,  and  engage  targets  -  MK19  grenade  launcher  machine  gun. 

9.  Maintain,  operate,  and  engage  targets  -  hand  grenades. 

10.  Maintain,  operate,  and  engage  targets  -  shotgun. 

11.  Maintain,  operate,  and  engage  targets  -  light  machine  guns  (M240B,  M249  squad 
automatic  weapon  (SAW),  M60). 

12.  Maintain,  operate,  and  engage  targets  -  M2,  .50  caliber  machine  gun. 

13.  Maintain,  operate,  and  engage  targets  -  night  vision  sights,  aiming  lights. 

14.  Employ  non-lethal  weapons  (NLW)  -  malodorants,  irritants,  pulsed  energy,  anti-traction, 
thermobaric,  special  munitions,  batons. 

Supervise,  Lead,  and  Train  Others 

1.  Conduct  inspections  and  perform  on-the-spot  corrections. 

2.  Conduct/lead  -  drill  and  ceremonies,  formations,  troop  movements. 

3.  Counsel  subordinates/peers. 

4.  Train  subordinates  to  perform  an  individual  task. 

5.  Prepare  for  and  conduct  preparatory  marksmanship  training  (PMT). 

6.  Conduct  performance  oriented  training. 

7.  Prepare  for  and  conduct  crew/battle  drills  training. 

8.  Prepare  a  training  outline/lesson  plan. 
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Knowledge  List 


Rules,  Regulations,  Laws  of  Conduct 

1.  Code  of  Conduct  -  Six  articles  of  required  or  prohibited  behaviors. 

2.  Laws  of  War  -  prohibited  acts,  prohibited  weapons/devices,  seizure/use  of  property. 

3.  Geneva  Convention  -  definition/treatment  of  prisoner  of  war  (PW)/detainees,  protection 
of  civilians. 

4.  Requirements  of  Federal,  State,  Local  Environmental  Regulations  -  spills,  hazardous 
materials,  marking,  reporting. 

5.  Rules  of  Engagement  (ROE)  and  Use  of  Force  -  fundamental  concept,  definitions,  self- 
defense,  RAMP  (return,  anticipate,  measure,  protect). 

6.  Requirements  of  the  Uniform  Code  of  Military  Justice  (UCMJ)  -  application,  selected 
punitive  articles,  definitions,  types  of  judicial/non-judicial  actions. 

Army  Polices  and  Practices 

1.  Drug  and  Alcohol  Abuse  Policy  -  prohibitions,  rehabilitation,  punishments,  voluntary 
help. 

2.  Equal  Opportunity  Policy  -  who,  where  applied,  complaint  process. 

3.  Sexual  Harassment  and  Reporting  Policy  -  definitions,  verbal,  non-verbal,  physical, 
assertive  behavior,  reporting. 

4.  Homosexual  Policy  -  don’t  ask,  don’t  tell,  don’t  harass;  orientations  vs.  conduct, 
reporting,  service  policies. 

5.  Army  Physical  Fitness  Standards  -  components,  conditions.  Army  Physical  Fitness  Test 
(APFT),  nutrition. 

6.  Interactions  With  News  Media  -  identification,  operational  security  (OPSEC),  rights, 
guidelines. 

Leadership  Procedures  and  Principles 

1.  Leadership  duties,  responsibilities,  authority,  and  requirements  of  officers,  warrant 
officers  (WO),  and  non-commissioned  officers  (NCO). 

2.  Principles  of  discipline. 

3.  Policies  and  procedures  of  the  chain  of  command  and  the  NCO  support  channel. 

4.  Principles  of  BE,  KNOW,  DO. 

5.  Steps  of  troop  leading  procedures  (TLP). 

6.  Steps  of  decision-making  (problem  solving). 

7.  Roles  and  responsibilities  of  NCO  in  training. 

Army  History,  Customs,  Values 

1.  Army  History  -  major  wars.  Cold  War,  All- Volunteer  Army,  organizations,  notable 
personalities. 

2.  History  of  the  NCO  -  evolution  of  ranks/responsibilities. 

3.  Army  Customs  and  Courtesies  -  uniform,  saluting/reporting,  bugle  calls,  ranks/insignia. 

4.  Army  Values  and  Warrior  Ethos. 
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Appendix  G 

Descriptive  Statistics  for  Task  and  Knowledge  Statement  Ratings 


Table  G.l  Mean  Time  Spent  Ratings  for  Task  Statements _ 

Total 


Incumbent  Supervisor  Sample 


First  Aid 

M 

SD 

M 

SD 

M 

SD 

Evaluate  Casualty 

3.40 

1.11 

3.38 

1.16 

3.39 

1.11 

Perform  FA  -  heat  injuries 

3.02 

1.30 

3.05 

1.13 

3.03 

1.25 

Perform  FA  -  bleeding  extremities 

2.96 

1.17 

3.05 

1.07 

2.99 

1.14 

Perform  FA  -  open  wound 

2.93 

0.98 

3.10 

1.09 

2.99 

1.01 

Put  on  field  dressing 

2.90 

1.02 

3.14 

1.21 

2.97 

1.07 

Request  MEDEVAC 

2.65 

1.27 

3.38 

1.53 

2.88 

1.39 

Perform  FA  -  shock 

2.82 

1.10 

3.00 

1.05 

2.87 

1.08 

Evacuate  from  vehicle 

2.81 

1.14 

2.90 

1.34 

2.84 

1.19 

Practice  preventive  med 

2.67 

1.20 

3.05 

1.32 

2.79 

1.24 

Initiate  IV 

2.61 

1.20 

3.13 

0.89 

2.75 

1.14 

Transport  casualty 

2.56 

1.20 

3.19 

1.54 

2.75 

1.33 

Perform  FA  -  fracture 

2.69 

1.08 

2.86 

0.99 

2.75 

1.05 

Perform  FA  -  bums 

2.72 

1.06 

2.73 

1.20 

2.72 

1.10 

Perform  mouth-to-mouth 

2.72 

1.03 

2.43 

1.03 

2.63 

1.03 

Perform  FA  -  throat  obstruction 

2.63 

1.02 

2.50 

1.05 

2.59 

1.03 

Recover  remains 

2.39 

1.12 

2.71 

1.14 

2.49 

1.12 

Buddy  Aid 

2.56 

1.18 

1.89 

1.23 

2.36 

1.23 

Perform  FA  -  cold  injuries 

NBC  Protection  & 
Decontamination 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

Put  on/wear  MOPP 

2.76 

1.34 

2.65 

1.14 

2.73 

1.27 

Decontaminate  self 

2.65 

1.14 

2.37 

1.12 

2.57 

1.13 

Detect  chemical  agents 

2.60 

1.16 

2.41 

1.12 

2.55 

1.14 

React  to  attack/hazard 

2.59 

1.21 

2.45 

1.19 

2.55 

1.19 

Exchange  MOPP 

2.52 

1.15 

2.53 

1.07 

2.52 

1.12 

Maintain  mask 

2.43 

1.19 

2.71 

1.06 

2.51 

1.15 

Cross  contaminated  area 

2.52 

1.09 

2.39 

1.14 

2.48 

1.10 

Decontaminate  equip 

Vehicle  &  Convoy  Tactical  Ops 

2.46 

1.11 

2.24 

1.25 

2.40 

1.14 

Perform  vehicle  PMCS 

3.25 

1.31 

3.77 

1.07 

3.41 

1.26 

Prepare  for  convoy 

3.15 

1.20 

3.48 

1.17 

3.25 

1.19 

Secure  vehicles  at  halt 

3.04 

1.08 

3.36 

1.40 

3.14 

1.19 

Perform  exit/enter  battle  drill 

2.96 

1.10 

3.05 

1.56 

2.99 

1.25 

Perform  forced  stop  battle  drill 

2.84 

1.10 

3.29 

1.35 

2.98 

1.19 

Provide  convoy  security 

2.81 

1.44 

3.23 

1.38 

2.94 

1.42 

Perform  break  contact  drill 

2.73 

1.12 

3.40 

1.23 

2.94 

1.18 

Drive  vehicle 

2.72 

1.21 

3.33 

1.28 

2.93 

1.26 

Occupy  floating  rally  pt 

Threat  ID  &  Reaction 

2.56 

1.10 

2.83 

1.29 

2.65 

1.16 

Move  under  direct  fire 

2.88 

1.03 

3.50 

0.83 

3.06 

1.01 
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Total 

Incumbent  Supervisor _ Sample 


M 

SD 

M 

SD 

M 

SD 

React  to  ambush  (mounted) 

2.79 

1.07 

3.55 

1.10 

3.01 

1.13 

React  to  contact 

2.80 

1.28 

3.38 

1.47 

2.99 

1.35 

ID  threats  visually 

2.67 

1.33 

3.32 

1.29 

2.88 

1.34 

React  to  ambush  (dismounted) 

2.71 

1.05 

3.28 

1.07 

2.86 

1.08 

Mark/report  UXO 

2.84 

1.10 

2.81 

1.03 

2.83 

1.07 

React  to  sniper 

2.64 

1.18 

3.15 

1.35 

2.80 

1.25 

Locate  mines  by  probing 

2.73 

0.96 

2.75 

1.00 

2.74 

0.96 

Conduct  self-extract 

2.66 

1.21 

2.58 

1.22 

2.63 

1.20 

Force  Protection 

Op  vehicle/personnel  checkpoint 

2.90 

1.28 

3.25 

1.29 

3.00 

1.28 

Conduct  intrusion  prevention 

2.81 

1.16 

3.11 

1.15 

2.90 

1.16 

Constr  uct  non-explosive  approach 

2.70 

1.29 

3.00 

1.50 

2.79 

1.35 

Conduct  surveillance 

2.53 

1.18 

3.32 

1.16 

2.78 

1.22 

Conduct  urban  patrol 

2.80 

1.05 

2.72 

1.67 

2.77 

1.25 

Visually  id  vehicles  &  aircraft 

2.61 

1.16 

2.95 

1.43 

2.72 

1.25 

Process  capturecFsuspicious 
material 

2.60 

0.88 

2.90 

1.25 

2.70 

1.01 

Id/sccure/etc.  CI/EPWs/etc. 

2.55 

0.93 

2.95 

1.28 

2.67 

1.05 

Conduct  ECP  ops 

2.51 

1.30 

3.00 

1.30 

2.66 

1.31 

Conduct  crowd  control 

2.49 

1.08 

3.00 

1.06 

2.64 

1.09 

React  to  man-to-man  contact 

2.61 

1.06 

2.58 

1.17 

2.60 

1.09 

Establish  &  occupy  OP 

2.43 

1.19 

2.80 

1.32 

2.55 

1.24 

Recognize  &  proc  intelligence 
items 

2.28 

0.88 

3.12 

1.11 

2.53 

1.02 

Urban  Ops,  Camouflage 

Move  over  obstacles  (dismounted) 

2.70 

0.93 

3.28 

1.18 

2.87 

1.03 

Search/secure  bldg/room 

2.77 

1.13 

2.82 

1.38 

2.78 

1.19 

Practice  noise/light/etc  disc. 

2.60 

1.20 

3.05 

1.20 

2.74 

1.21 

Conduct  bldg  assaults 

2.51 

1.10 

3.21 

1.05 

2.70 

1.12 

Select/etc.fighting  positions 

2.50 

1.11 

3.11 

1.08 

2.67 

1.13 

Perform  firc/maneuver  during  urban 
ops 

2.56 

0.93 

2.83 

1.15 

2.64 

1.00 

Execute  bldg  breach 

2.41 

0.96 

3.07 

1.00 

2.59 

1.00 

Prep  individual/crcw  weapons 
positions 

2.45 

1.11 

2.89 

1.41 

2.58 

1.21 

Prep  defensive  pos  range  card 

2.41 

0.91 

2.68 

1.21 

2.51 

1.03 

Clear  fields  of  fire 

2.40 

0.88 

2.58 

1.22 

2.45 

0.99 

Camouflage  equipment 

2.24 

0.98 

2.67 

1.32 

2.38 

1.11 

Camouflage  self 

2.32 

0.93 

2.29 

1.23 

2.31 

1.03 

Navigation,  Communication,  Call  for  Fires 
Setup/op  tactical  radios  2.49 

1.08 

3.81 

0.93 

2.92 

1.20 

Navigate  from  one  pt  to  another 

2.52 

1.09 

3.57 

1.12 

2.86 

1.20 

Deter  direction/location 

2.55 

0.92 

3.20 

1.36 

2.76 

1.11 

Perform  voice  communications 

2.23 

1.24 

3.73 

1.24 

2.73 

1.42 

Locate  target  by  grid  coordinates 

2.42 

1.10 

3.30 

1.13 

2.69 

1.17 

Call  for/adjust  indirect  fire 

2.18 

0.98 

3.25 

1.24 

2.48 

1.16 

Construct  field  exp  antennae 

2.16 

0.93 

2.94 

1.06 

2.42 

1.03 

Use  visual  signalling  tech 

2.15 

1.00 

2.65 

1.35 

2.30 

1.13 
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Incumbent 

Supervisor 

Weapons,  Grenades,  Mines, 

M 

SD 

M 

SD 

M 

SD 

M16/M4 

3.06 

1.13 

3.73 

1.16 

3.25 

1.18 

M9 

2.95 

1.04 

3.44 

1.46 

3.11 

1.20 

Light  machine  guns 

2.57 

1.13 

3.10 

1.48 

2.74 

1.27 

M2 

2.49 

1.01 

3.11 

1.41 

2.70 

1.19 

Night  vision  sights 

2.43 

1.12 

3.10 

1.41 

2.67 

1.26 

Employ  NLW 

2.39 

0.92 

3.27 

1.27 

2.64 

1.09 

Shotgun 

2.24 

1.12 

3.55 

1.13 

2.60 

1.26 

M19 

2.53 

1.19 

2.72 

1.56 

2.60 

1.32 

M203 

2.27 

1.02 

2.90 

1.33 

2.49 

1.17 

Employ/recover  antitank  mines 
Employ/recover  antipersonnel 

2.33 

0.88 

2.75 

1.22 

2.45 

0.99 

mines 

2.29 

1.24 

2.79 

1.31 

2.44 

1.27 

Employ  demolitions 

2.24 

0.99 

2.60 

1.51 

2.33 

1.13 

M136 

2.10 

0.97 

2.71 

1.49 

2.26 

1.15 

Hand  grenades 

Supervise,  Lead,  Train  Others 

2.12 

0.92 

2.50 

1.50 

2.23 

1.13 

Conduct  inspections 

2.61 

1.16 

3.70 

1.22 

2.94 

1.28 

Train  subordinates 

2.63 

1.12 

3.65 

1.09 

2.93 

1.20 

Counsel  subordinates 

2.56 

0.96 

3.60 

0.99 

2.89 

1.08 

Prepare  for/conduct  PMT 

2.60 

1.16 

3.20 

1.28 

2.79 

1.22 

Conduct  Perform-oriented  training 

2.63 

1.16 

3.14 

1.24 

2.78 

1.20 

Prepare  for/conduct  drills 

2.45 

1.00 

3.40 

1.19 

2.75 

1.14 

Conduct/lead  drill/ceremonies 

2.58 

1.01 

2.90 

1.48 

2.68 

1.18 

Prepare  training  outline 

2.39 

0.95 

3.05 

1.32 

2.59 

1.11 

Note.  Incumbent  n  =  28  -  53,  supervisor  n  =  10  -  22. 
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Table  G.2.  Mean  Test  Emphasis  Ratings  for  Task  Statements 


Total 

Incumbent  Supervisor _ Sample 

M  SD  M  SD  M  SD 


First  Aid 

Evaluate  Casualty 
Request  MEDEVAC 
Perform  FA  -  heat  injuries 
Perform  FA  -  bleeding  extremities 
Perform  FA  -  open  wound 
Practice  preventive  med 
Put  on  field  dressing 
Initiate  IV 
Perform  FA  -  shock 
Perform  FA  -  cold  injuries 
Perform  FA  -  burns 
Transport  casualty 
Perform  mouth-to-mouth 
Perform  FA  -  fracture 
Evacuate  from  vehicle 
Perform  FA  -  throat  obstruction 
Recover  remains 
Buddy  Aid 

NBC  Protection  &  Decontamination 

Decontaminate  self 

React  to  attack/hazard 

Exchange  MOPP 

Put  on/wear  MOPP 

Detect  chemical  agents 

Cross  contaminated  area 

Maintain  mask 

Decontaminate  equip 

Vehicle  &  Convoy  Tactical  Ops 

Perform  vehicle  PMCS 

Secure  vehicles  at  halt 

Prepare  for  convoy 

Perform  exit/enter  battle  drill 

Perform  forced  stop  battle  drill 

Perform  break  contact  drill 

Drive  vehicle 

Provide  convoy  security 

Occupy  floating  rally  pt 

Threat  ID  &  Reaction 

ID  threats  visually 

React  to  contact 

React  to  ambush  (mounted) 

Move  under  direct  fire 
React  to  ambush  (dismounted) 
Mark/report  UXO 
React  to  sniper 


3.33 

1.11 

3.77 

1.11 

3.46 

1.14 

2.81 

1.27 

4.23 

0.97 

3.23 

1.36 

3.26 

1.09 

3.14 

0.99 

3.23 

1.06 

3.06 

1.18 

3.55 

1.01 

3.21 

1.14 

3.02 

1.01 

3.41 

1.26 

3.14 

1.10 

2.88 

1.14 

3.67 

1.02 

3.11 

1.14 

2.90 

1.04 

3.59 

1.01 

3.11 

1.08 

2.86 

1.34 

3.63 

0.76 

3.10 

1.24 

2.96 

1.06 

3.09 

1.11 

3.00 

1.08 

3.00 

1.10 

2.90 

0.89 

2.97 

1.07 

2.87 

1.12 

3.18 

1.14 

2.96 

1.15 

2.67 

1.23 

3.64 

1.29 

2.96 

1.33 

2.73 

0.99 

3.23 

0.92 

2.88 

1.01 

2.79 

0.99 

3.05 

1.20 

2.86 

1.13 

2.81 

1.11 

2.90 

1.34 

2.84 

1.19 

2.83 

1.13 

2.82 

1.22 

2.82 

1.17 

2.52 

1.34 

3.05 

1.32 

2.69 

1.34 

2.66 

1.07 

2.50 

1.19 

2.61 

1.08 

2.85 

0.99 

2.71 

1.05 

2.81 

0.99 

2.84 

1.06 

2.63 

1.01 

2.79 

1.03 

2.88 

1.03 

2.44 

1.10 

2.77 

1.09 

2.88 

1.07 

2.45 

1.15 

2.76 

1.11 

2.75 

0.99 

2.67 

1.14 

2.73 

1.09 

2.82 

1.04 

2.44 

1.15 

2.72 

1.07 

2.72 

1.24 

2.67 

1.11 

2.70 

1.18 

2.71 

1.02 

2.44 

1.15 

2.64 

1.04 

3.15 

1.01 

4.18 

0.66 

3.45 

1.03 

3.19 

1.00 

4.00 

0.98 

3.43 

1.06 

3.08 

0.97 

4.23 

0.75 

3.41 

1.07 

3.08 

1.08 

4.18 

0.85 

3.41 

1.12 

3.08 

1.14 

4.05 

1.21 

3.37 

1.23 

2.88 

1.22 

4.23 

0.87 

3.28 

1.26 

2.91 

1.19 

3.95 

1.13 

3.25 

1.26 

2.81 

1.12 

4.00 

0.93 

3.16 

1.24 

2.80 

0.98 

3.65 

1.09 

3.06 

1.07 

3.43 

1.31 

4.18 

0.80 

3.66 

1.22 

3.20 

1.21 

4.18 

0.85 

3.50 

1.22 

3.21 

1.30 

3.91 

1.11 

3.42 

1.27 

3.15 

1.21 

3.68 

0.95 

3.31 

1.16 

3.09 

1.21 

3.82 

0.85 

3.31 

1.20 

3.15 

1.15 

3.36 

1.00 

3.22 

1.11 

2.86 

1.35 

3.82 

0.91 

3.15 

1.29 
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Incumbent  Supervisor 


M 

SD 

M 

SD 

Locate  mines  by  probing 

3.02 

1.25 

2.89 

1.15 

Conduct  self-extract 

2.94 

1.32 

3.05 

1.32 

Force  Protection 

Op  vehicle/personnel  checkpoint 

3.17 

1.29 

3.77 

1.11 

React  to  man-to-man  contact 

3.16 

1.24 

3.48 

1.33 

Visually  id  vehicles  &  aircraft 

3.12 

1.29 

3.32 

1.25 

Conduct  urban  patrol 

3.08 

1.24 

3.36 

1.62 

Conduct  intrusion  prevention 

2.94 

1.24 

3.64 

1.00 

Recognize  &  proc  intelligence  items 

2.81 

1.13 

3.81 

0.93 

Conduct  ECP  ops 

2.75 

1.44 

3.73 

1.08 

Construct  non -explosive  approach 

2.85 

1.21 

3.41 

1.30 

Conduct  surveillance 

2.79 

1.36 

3.55 

1.01 

Id/secure/etc.  CI/EPWs/etc. 

2.84 

1.09 

3.32 

1.46 

Process  captured/suspicious  material 

2.78 

0.99 

3.41 

1.14 

Establish  &  occupy  OP 

2.79 

1.30 

3.36 

1.18 

Conduct  crowd  control 

2.71 

1.22 

3.29 

1.01 

Urban  Ops,  Camouflage 

Search/secure  bldg/room 

3,22 

1.29 

3.37 

1.26 

Prepare  individual/crew  weapon  positions 

3.00 

1.31 

3.57 

1.16 

Conduct  bldg  assaults 

3.00 

1.26 

3.58 

0.96 

Practice  noise/light/etc  disc. 

2.90 

1.30 

3.68 

0.78 

Perform  fire/maneuver  during  urban  ops 

2.94 

1.09 

3.48 

1.03 

Select/etc.fighting  positions 

2.92 

1.06 

3.43 

0.93 

Execute  bldg  breach 

2.84 

1.21 

3.55 

1.05 

Move  over  obstacles  (dismounted) 

3.00 

1.09 

3.19 

1.21 

Clear  fields  of  fire 

2.90 

1.07 

3.23 

0.92 

Camouflage  equipment 

2.80 

1.08 

3.00 

1.05 

Prepare  defensive  pos  range  card 

2.65 

1.09 

3.00 

1.00 

Camouflage  self 

2.63 

1.04 

2.90 

0.97 

Navigation,  Communication,  Call  for  Fires 

Navigate  from  one  pt  to  another 

3.06 

1.27 

4.00 

0.77 

Locate  target  by  grid  coordinates 

3.10 

1.17 

3.68 

1.04 

Setup/op  tactical  radios 

2.92 

1.32 

3.86 

0.89 

Deter  direction/location 

2.92 

1.21 

3.82 

1.01 

Call  for/adjust  indirect  fire 

2.80 

1.23 

3.62 

1.16 

Perform  voice  communications 

2.65 

1.24 

3.82 

0.91 

Use  visual  signalling  tech 

2.75 

1.27 

3.29 

1.06 

Construct  field  exp  antennae 

2.54 

1.31 

3.55 

0.83 

Weapons,  Grenades,  Mines, 

M16/M4 

3.45 

1.04 

3.95 

1.09 

Night  vision  sights 

3.14 

1.23 

4.14 

0.65 

M9 

3.11 

1.32 

3.95 

1.10 

Light  machine  guns 

3.00 

1.28 

4.05 

0.80 

M19 

3.05 

1.14 

3.75 

1.07 

M2 

2.85 

1.43 

4.10 

0.91 

Shotgun 

2.83 

1.35 

3.71 

1.16 

M203 

2.71 

1.16 

3.76 

0.70 
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Total 
Sample 
M  SD 

2.98  1.24 

2.97  1.33 

3.35  1.29 

3.25  1.30 

3.18  1.27 

3.16  1.40 

3.15  1.21 

3.12  1.17 

3.04  1.39 

3.03  1.26 

3.03  1.27 

2.99  1.22 

2.97  1.07 

2.97  1.27 

2.89  1.19 

3.26  1.28 

3.17  1.26 

3.17  1.21 

3.14  1.22 

3.10  1.09 

3.07  1.03 

3.06  1.20 

3.06  1.10 

3.00  1.03 

2.86  1.08 
2.76  1.05 

2.71  1.00 

3.34  1.23 

3.27  1.19 

3.21  1.29 

3.20  1.24 

3.04  1.27 

3.01  1.25 

2.90  1.22 

2.85  1.24 

3.60  1.09 

3.47  1.21 

3.38  1.33 

3.32  1.29 

3.27  1.17 

3.23  1.43 

3.09  1.39 

3.05  1.16 


Incumbent 

Supervisor 

Total 

Sample 

M 

SD 

M 

SD 

M 

SD 

Employ  NLW 

2.74 

1.20 

3.65 

0.93 

3.02 

1.21 

Hand  grenades 

2.72 

1.23 

3.19 

1.25 

2.87 

1.26 

M136 

2.42 

1.24 

3.68 

1.00 

2.78 

1.32 

Employ/recover  antipersonnel  mines 

2.62 

1.31 

2.89 

1.20 

2.70 

1.26 

Employ  demolitions 

2.49 

1.18 

2.88 

1.20 

2.60 

1.20 

Employ/recover  antitank  mines 

2.53 

1.09 

2.76 

1.35 

2.60 

1.18 

Supervise,  Lead,  Train  Others 

Conduct  Perform-oriented  training 

3.00 

1.03 

4.00 

1.02 

3.31 

1.14 

Prepare  for/conduct  PMT 

2.85 

1.15 

4.10 

0.77 

3.24 

1.25 

Conduct  inspections 

2.83 

1.37 

4.10 

0.83 

3.22 

1.36 

Train  subordinates 

2.86 

1.15 

4.05 

0.80 

3.21 

1.21 

Prepare  for/conduct  drills 

2.90 

1.23 

3.86 

0.89 

3.20 

1.19 

Prepare  training  outline 

2.71 

1.26 

3.95 

1.00 

3.10 

1.31 

Counsel  subordinates 

2.56 

0.97 

3.95 

1.16 

2.99 

1.22 

Conduct/lead  drill/ceremonies 

2.68 

1.19 

3.52 

1.25 

2.93 

1.28 

Note.  Incumbent  n  =  39  -  53,  supervisor  n=  16-22. 


G-6 


Table  G.3.  Mean  Supervisor  Test  Emphasis  Ratings  for  Knowledge  Statements 

M  SD 


Rules,  Regulations,  Laws 

Rules  of  Engagement 

4.10 

1.18 

Universal  Code  of  Military  Justice 

3.86 

1.20 

Laws  of  War 

3.76 

1.37 

Geneva  Convention 

3.71 

1.31 

Code  of  Conduct 

3.29 

1.55 

Requirements  of  Environ  Regulations 

3.05 

1.50 

Army  Policies  &  Practices 

Interactions  with  Media 

4.00 

1.00 

Physical  Fitness  Stnds 

3.76 

1.14 

Drug  &  Alcohol  Abuse 

3.62 

1.20 

Sexual  Harassment 

3.24 

1.22 

Equal  Opportunity 

3.19 

1.36 

Homosexual  Policy 

3.15 

1.23 

Leadership  Procedures 

Leadership  duties 

4.48 

0.68 

Steps  of  TLP 

4.24 

0.70 

Roles  of  NCO  in  training 

4.14 

0.79 

BE,  KNOW,  DO 

4.10 

0.83 

Policies  chain  of  command 

4.05 

0.86 

Steps  of  decision-making 

4.05 

1.16 

Principles  of  discipline 

3.90 

1.09 

Army  History,  Customs,  Values 
Customs  and  courtesies 

3.81 

0.93 

History  of  NCO 

3.67 

1.02 

Army  values 

3.62 

1.07 

Army  history 

3.48 

1.17 

Note.  n  =  21. 
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